TensorRT Batch Inference: different results

[spolisetty]

@Htut,

Sorry for the delayed response. Are you using dynamic shape input ? as you are working with batches, Please make sure you’re marking dynamic shapes correctly. Looks like you’re also not mentioning input layer name and dynamic shapes. Please refer following,

Work with batch in TensorRT

Hi @v.stadnichuk, We noticed that in trtexec command you’re using input node name wrongly. When we visualize onnx file using netron identified that it has to be input_1. Please try with updated trtexec command. trtexec --explicitBatch --onnx=apm_one_input.onnx --minShapes=input_1:1x64x64x3 --optShapes=input_1:20x64x64x3 --maxShapes=input_1:100x64x64x3 --saveEngine=apm_one_input.plan Thank you.

https://github.com/NVIDIA/TensorRT/blob/master/samples/opensource/trtexec/README.md#example-4-running-an-onnx-model-with-full-dimensions-and-dynamic-shapes