Different between context->enqueue, enqueueV2, enqueueV3

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

A clear and concise description of the bug or issue.

Environment

TensorRT Version: 8
GPU Type: 2080Ti
Nvidia Driver Version: 470
CUDA Version: 11.4
CUDNN Version:
Operating System + Version:
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

Can anyone explain for me: different between context->enqueue, enqueueV2, enqueueV3 when use tensorrt inference. Which solution should I use?

4

It appears all others except v3 are deprecated in the latest version TensorRT: nvinfer1::IExecutionContext Class Reference, but I don’t have any insight into why it was changed. I suppose the v3 API must be preferable in some way.

5

Hi @vuminhduc9755 ,
enqueue: oldest api, support implicit batch, is deprecated.
enqueueV2: replacement of enqueue, support explict batch. is deprecated now.
enqueueV3: latest api, support data dependent shape, recommend to use now.

Please check TensorRT: nvinfer1::IExecutionContext Class Reference for details.

Thanks