TF-TRT How to extract quantized weights after post-training quantization

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hello everyone,

I am using TensorRT5 in top of Tensorflow.
I am applying INT8 quantization on my trained model and I am getting INT8 TRT graph.

I want to extract and read the quantized weights that the TRT graph uses.

Do you have any idea how we can read the weights used for the INT8 inference/graph?

Thanks,

2

Hi,

Please refer below link to extract the TRT calibration table after calibration is done:
https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html#post-train

Thanks

3

Hello,

The calibration table gives only the threshold T used to quantize to the activations.

I want to extract the weights after quantization not the this threshold!!

Thanks.

4

Hi,

API to extract the weights after quantization is currently not supported in TRT.

Thanks