Why we need to use setDynamicRange() for all layer data in TensorRT

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Why we need to use setDynamicRange() for all layer data? It can improve the inference performance in TensorRT?

5

Hi @xuwenyao001,

TensorRT tactic selection is profile based implementation, so we need min/opt/max in each profile for timing purpose. TensorRT needs the dynamic-range for each activation tensor in the network.
We use setDynamicRange API to manually set the dynamic range for each network tensor
Or
We use INT8 calibration to generate per tensor dynamic range using the calibration dataset.

Please refer user guide for more details.

Thank you.