Hi all, I want to know how the existing sample code can be modified to infer with batch size more than 1. In the existing sample code from the directory /usr/src/tensorrt/samples/python/network_api_pytorch_mnist/sample.py and model.py the inference on gpu will be done with only one image (784, ).…

How to infer more than one image (batch size > 1)

SivaRamaKrishnaNV October 4, 2023, 3:01am 4

Dear @trivedi.nagaraj,
Could you confirm if the platform is Jetson Nano or Jetson Orin Nano?

I see the sample is using EXPLICIT_BATCH . Did you check with INPUT_SHAPE = (2, 1, 28, 28) in sample.py for batchsize 2?

Topic		Replies	Views
Jetson Xavier - Inference multiple images Jetson AGX Xavier	7	1024	October 18, 2021
TensorRT running inference with batch size > 1 TensorRT tensorrt	8	3711	January 26, 2021
Why does my batchsize affect the output of my inference function? Jetson AGX Orin jetson-inference	5	55	December 16, 2024
Tensorrt inference on multiple batches TensorRT tensorrt , jetson-inference	5	3181	October 27, 2022
Batch Inference Wrong in Python API TensorRT	15	3562	October 12, 2021
Batch execution of trt model TensorRT cudnn	1	448	February 29, 2024
TensorRT Batch Inferences : empty outputs TensorRT tensorrt , jetson-inference	8	1956	July 18, 2024
Tensorrt7 batchsize infer results problems GPU-Accelerated Libraries tensorrt	0	439	March 20, 2020
Run inference on a batch of images & parallel inference using cuda on python threads TensorRT tensorrt , cuda	6	2333	January 6, 2022
Batch inference doesn't work on Nano? Jetson Nano	2	1026	October 14, 2021