TensorRT 5 batch size


I want to perform inference from caffe models with different batch size and compare results. How can I do that? In the caffe_resnet50.py sample it seems that images are being loaded one at a time and I’m not sure how to modify the code to achieve what I want. Can you help me out?


Try looking at the int8_caffe_mnist.py sample in a similar directory to the caffe_resnet50.py sample. You can read more about it here: https://docs.nvidia.com/deeplearning/sdk/tensorrt-sample-support-guide/index.html#int8_caffe_mnist.

If you’re using an NGC TensorRT Container, it will be located here:


This code provides an example of adding batch_size when creating the engine and doing inference.

NVIDIA Enterprise Support