Why is the variable `nbins` in `_compute_amax_entropy` determined by the sign?

dk.hong · March 9, 2023, 12:33am

As I understand, HistogramCalibrator collects data distribution of each activation, and it is compressed into 2^bits bins for computing entropy.

In the signed 8-bit setting, integer values among -128~127 can be used, and the number of candidate integers is 256.

In the unsigned 8-bit setting, integer values among 0~255 can be used, and the number of candidate integers is also 256.

So, I think nbins in _compute_amax_entropy should always be 256 regardless of the sign.

However, the implementation in pytorch_quantization, nbins of signed 8-bit setting is half that of unsigned 8-bit setting.

github.com

NVIDIA/TensorRT/blob/8e756f163f83d54389c7ff82235e57a518f6eb03/tools/pytorch-quantization/pytorch_quantization/calib/histogram.py#L204


      
                  distr = distr / summ
          
          
bins = calib_hist[:]
          bins[0] = bins[1]
          
          
total_data = np.sum(bins)
          
          
divergences = []
          arguments = []
          
          
# we are quantizing to 128 values + sign if num_bits=8
          nbins = 1 << (num_bits - 1 + int(unsigned))
          
          
starting = start_bin
          stop = len(bins)
          
          
new_density_counts = np.zeros(nbins, dtype=np.float64)
          
          
for i in range(starting, stop + 1, stride):
              new_density_counts.fill(0)
              space = np.linspace(0, i, num=nbins + 1)

Can you inform me what I missed?