Multi Stream in TensorRT


A clear and concise description of the bug or issue.


TensorRT Version -> 7.0:
GPU Type -> RTX:
Nvidia Driver Version -> 440.64:
CUDA Version -> 10.2:
CUDNN Version -> 7.6.5:
Operating System + Version -> Ubuntu 18.04:
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag) ->

I want to ask about any resources which guides how to implement multiple streams in parallel on a single engine. I am able to create an engine with a batch size > 1. But I have no idea how to process multiple stream (=batch size) in parallel with single engine!

Hi @y14uc339,
Hope these links helps you