Building int8 model: combining direct dynamic range setting with calibration

Hi,

I’m building a TensorRT engine for int8 precision, and have to use implicit quantization because I’m targeting the DLA. Is it possible to combine direct setting of dynamic range (via tensor.set_dynamic_range()) with calibration? There are some tensors that I know the dynamic range for, and can set a tighter range than I’d get through calibration, but for the rest of the tensors I want the dynamic range to come from the calibration data. In particular, if I directly set the dynamic range of some tensors using tensor.set_dynamic_range(), will that value be wiped out if I subsequently run calibration?

Thanks!

To answer my own question, it looks like you CAN combine direct precision setting and calibration. If you set the dynamic range manually, it will NOT be over-written by calibration. Seeing output like:

11/19/2024-12:41:44] [TRT] [V] User overriding scale with scale and zero-point Quantization(scale: {0.0472441,}, zero-point: {0,})
[11/19/2024-12:41:44] [TRT] [V] User overriding scale with scale and zero-point Quantization(scale: {0.0472441,}, zero-point: {0,})
[11/19/2024-12:41:44] [TRT] [V] INT8 Inference Tensor scales and zero-points: /network/backbone/maxpool/MaxPool_output_0 scale and zero-point Quantization(scale: {0.0472556,}, zero-point: {0,})
[11/19/2024-12:41:44] [TRT] [V] User overriding scale with scale and zero-point Quantization(scale: {0.0472441,}, zero-point: {0,})
[11/19/2024-12:41:44] [TRT] [V] User overriding scale with scale and zero-point Quantization(scale: {0.0472441,}, zero-point: {0,})

So it seems to be respecting the user precision override.