[TensorRT] batch vs shapes

I have exported my Torch model with the following command:

        model, dummy_input, outfile, verbose=verbose,
        input_names=['input_batch'], output_names=output_names,
        # keep_initializers_as_inputs=True,
            'input_batch': {0: 'dynamic'},
            'cif': {0: 'dynamic'},
            'caf': {0: 'dynamic'},

The shape of input is, (1x3x369x641, CHW)

The saved model is openpifpaf_resnet50_641_369.onnx

Now, I want to convert it to TensorRT engine file, with the batch size of 32.

I won’t change the width and height of input, but am interested in batch inference.

Which one is the correct command to use?

trtexec --onnx=openpifpaf_resnet50_641_369.onnx \
    --verbose \
    --explicitBatch \
    --minShapes=input_batch:1x3x369x641 \
    --maxShapes=input_batch:32x3x369x641 \
    --optShapes=input_batch:32x3x369x641 \
    --saveEngine=openpifpaf-resnet50-dynamic_b32.onnx \
    --fp16 \
trtexec --onnx=openpifpaf_resnet50_641_369.onnx \
    --verbose \
    --explicitBatch \
    --batch=32 \
    --saveEngine=openpifpaf-resnet50-dynamic_b32_2.onnx \
    --fp16 \

Also, in trtexec command, what is a difference between batch and the batch inside a shape?


Please refer to the following document, which may help you:

Thank you.