TensorRT INT8 calibration

AI & Data Science Deep Learning (Training & Inference) TensorRT
, ,
1

Hey,

I am trying to optimise my custom model using INT8 precision. I have succeeded in optimising the same in FP16 but could not get the correct way to generate a calibrator for INT8. If anyone could suggest a good tutorial, it would be really helpful.

Thanks in advance.

1 Like
3

Hi @RohanShah,

Please refer

https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/sampleINT8

Thank you.

4

i already referred all these documents & github repo. but i can’t find anything helpful from which i can generate calibrator for my custom model. because there all are very confusing so if you have any video tutorial that will help me a lot.

thank you

5

If you are OK to use TensorRT python API, check out my Demo #6: Using INT8 and DLA core of tensorrt_demos. You should be able to reuse most of my calibrator.py code. And the code for building the INT8 TensorRT engine is here.

2 Likes
6

@jkjung13 thank you.