Quantization to int8 still confusing

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hello all,
I have just read the 8bit inference using tensorRT pdf, but still got stuck in this slice.
The bins are constructed from the histgram, right? However, in this slice, it seems each bin only contains one value, maybe the sum up of all values in the same histgram. I am not sure if this is the right understanding.
A follow up question is about the example below. How are this 3 and 4 obtained as indicated by the red arrow? What if I want to quantize 8 bins into 3 or 5 or 6 bins? How does tensorRT handle it?

image
image1497×1625 474 KB

I appreciate any help. Thanks.

2

Hi,
What’s in a bin is the number of scalars falling in its range, not sum of them. 3 and 4 are the number of non-empty bins.
Sorry, but quantiztion is TRT is handled by internal API/algorithm, which I am not aware of, so I am not sure how 8 bins can be quantized to 3 or 5 or 6 bins is handled in TRT.

Thanks