TensorRT 5 batch size

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hi.

I want to perform inference from caffe models with different batch size and compare results. How can I do that? In the caffe_resnet50.py sample it seems that images are being loaded one at a time and I’m not sure how to modify the code to achieve what I want. Can you help me out?

2

Hi,

Try looking at the int8_caffe_mnist.py sample in a similar directory to the caffe_resnet50.py sample. You can read more about it here: https://docs.nvidia.com/deeplearning/sdk/tensorrt-sample-support-guide/index.html#int8_caffe_mnist.

If you’re using an NGC TensorRT Container, it will be located here:

/workspace/tensorrt/samples/python/int8_caffe_mnist/sample.py

This code provides an example of adding batch_size when creating the engine and doing inference.

Thanks,
NVIDIA Enterprise Support