Tensorrt with implicit batchsize

[mayuning2nd]

Description

confused with the implict batch_size inference. I intend to improve the overall throughput of a cnn inference task.
in the documents, it suggest using batching .
but the api shows that batch is deprecated with enqueue function and enqueueV3 works only for Explicit mode. (if I did not use [NetworkDefinitionCreationFlag::kEXPLICIT_BATCH] flag , the engine failed to build)

Environment

TensorRT Version: 8.5.1
GPU Type: xavier nx
Nvidia Driver Version:
CUDA Version: 11.4
CUDNN Version: 8.6
Operating System + Version: n.a.
Python Version (if applicable): n.a.
TensorFlow Version (if applicable): n.a.
PyTorch Version (if applicable): n.a.
Baremetal or Container (if container which image + tag): n.a.

[mayuning2nd]

I exported the network with only batchsize (the first dim ) be the dynamic_axes. when I build the engine with different optimization profile , like 1/4/8 or 3/3/3 as the batch dimension. the output dim never updated and remains as 1xCxHxW even I already set the profile for the output tensor also; I expect at least for a 3/3/3 batch dimension profile, the output batch dim should be 3; any idea?

[mayuning2nd]

I figured out that is due to a pytorch bug. so to disturb;