How do I get the float value of the scale factor in TensorRT calibration table？

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

“relu46: 3c92aa10
relu47: 3c4f19cd
relu48: 3ca714f8
pool2: 3c24dc61
output: 3db74ce8”

these are several lines in my calibration table, It seems like the scale factor was written in hexadecimal, how can I get the float value in decimal? Expecting for the answer, thanks

2

Hi,

Considering HEX value as big endian:
CalibrationTable ----> Scale factor ----> Max range
relu46: 3c92aa10 ----> 0.0179033577 ----> 55.85 (1/0.0179033577)

Similarly mapping other values you will get the float values.

Thanks

3

I am so sorry, I am still not clear about how to get 0.0179033577 with 3c92aa10, could you say some steps to get the mapping(3c92aa10-> 0.0179033577) in detail? This table is a int8 calibration table. I have tried several ways, but all failed, such as reverse 3c92aa10 to 01aa29c3, then convert it to a integer(27929027), then devide 127 by the integer…

4

Hi,

You can use any HEX to Float converter to convert the 3c92aa10-> 0.0179033577.
Just make sure the HEX value is considered as big endian during conversion.

Thanks.

5

Thanks, I had got how it works.
But I still have another question, for example, the output of my network is a 1*10 vector, (f1, …, f10), where fi (i=1,…10) is a float number, could I use scale_factor to get a verctor of (i1 = f1/scale_factor, i2= f2/ scale_factor, …, i10=f10/scale_factor),where ii is a int8 number, with a small precision loss? Because for doing this, I can compute the similarity of two vectors quickly on CPU. thanks

6

Hi,

I haven’t tried that, it’s something that is handled internally by TRT algorithm.

Please refer to below link for more details regarding Quantized Networks:
https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#work-with-qat-networks

Hope this might help.

Thanks

7

OK, Thank you very much!