INT8 quantization

I recently tried the TF-TRT script for INT8 quantization. But, I did not get the calib_tables. The calib_table files are empty. I used automatic quantization of TF-TRT feature (using the calibrate function provide by the converter). I am unable to attach the frozen graph that Im trying. But I am wondering if there are any conditions to be met for calibration? (like a specific NVIDIA hardware or CUDA version) . I tried using the latest nvidia tensorflow container (19.10-py3).

Hi,
Can you provide the following information so we can better help?
Provide details on the platforms you are using:
o Linux distro and version
o GPU type
o Nvidia driver version
o CUDA version
o CUDNN version
o Python version [if using python]
o Tensorflow version
o TensorRT version
o If Jetson, OS, hw versions

Also, if possible please share the script that you are using for INT8 quantization.

Meanwhile, please look into below links for steps and sample examples:
https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html#int8-quantization
https://github.com/tensorflow/tensorrt/tree/master/tftrt/examples/image-classification

Thanks

The GPU I use is Quadro GV100 with driver version : 430.50 and CUDA version 10.1.

All other info is already provided when I said that I use the container “tensorflow:19.10-py3”, I hope.

I’ll soon check if I can share the script and the input graph with you. In the meantime, it would be nice if you provide sample scripts for simple examples (something like hello world). Then I might read the above example you mentioned.

Hi,

Please refer to the below link for examples and user guide:
https://github.com/tensorflow/tensorrt

https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html#int8-quantization

Thanks

I am aware of these things. I am asking for simple examples (e.g. one matmul). Once people set up those examples, then they can follow tougher ones like object detection or image classification. I know that it has become typical of vendors to put the ball in the customer’s court and ask them to do everything…I won’t be surprised if I get a response that “go ahead and test it for a simple matmul yourself”. But, it is the complex dependencies on hardware and software that’s causing the current situation. So, if you put a simple example (e.g. matmul) in the container and show steps how it works, then it’ll become clear to me how to extend it for my own examples.

Hi,
You can also refer to below test scripts for as additional samples:
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/python/compiler/tensorrt/test

Thanks