Multi Stream in TensorRT


A clear and concise description of the bug or issue.


TensorRT Version → 7.0:
GPU Type → RTX:
Nvidia Driver Version → 440.64:
CUDA Version → 10.2:
CUDNN Version → 7.6.5:
Operating System + Version → Ubuntu 18.04:
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag) →

I want to ask about any resources which guides how to implement multiple streams in parallel on a single engine. I am able to create an engine with a batch size > 1. But I have no idea how to process multiple stream (=batch size) in parallel with single engine!

Hi @y14uc339,
Hope these links helps you