TensorRT INT8 calibration

Hey,

I am trying to optimise my custom model using INT8 precision. I have succeeded in optimising the same in FP16 but could not get the correct way to generate a calibrator for INT8. If anyone could suggest a good tutorial, it would be really helpful.

Thanks in advance.

1 Like

Hi @RohanShah,

Please refer
https://docs.nvidia.com/deeplearning/tensorrt/sample-support-guide/index.html#int8_sample

https://docs.nvidia.com/deeplearning/tensorrt/sample-support-guide/index.html#int8_caffe_mnist

Thank you.

i already referred all these documents & github repo. but i can’t find anything helpful from which i can generate calibrator for my custom model. because there all are very confusing so if you have any video tutorial that will help me a lot.

thank you

If you are OK to use TensorRT python API, check out my Demo #6: Using INT8 and DLA core of tensorrt_demos. You should be able to reuse most of my calibrator.py code. And the code for building the INT8 TensorRT engine is here.

2 Likes

@jkjung13 thank you.