TensorRT --- non-int8 fallback when trying to calibrate ONNX model

[spolisetty]

The missing scales and zero-points for some tensors suggest that the calibration process might be incomplete. This could happen if the calibration data wasn’t representative of the actual data you’ll use for inference.

Please refer to the following, which may help you.

Getting warning message [TensorRT] WARNING: Missing scale and zero-point for tensor (Unnamed Layer* 0)

Hi all, I have modifying the sample.py code from the directory /usr/src/tensorrt/samples/python/network_api_pytorch_mnist/ to accommodate for INT8 calibration. Though it ran successfully, I got few warning messages related to the layers in the network created. Here is the output below, please refer to the WARNING messages. I am unable to understand, why it has come? is it acceptable?. If not acceptable then what could be the solution to avoid these warning messages. Please help me in this r…

@razvanciubotaru21

Then I am trying to run inference using the INT8 engine file in Deepstream 6.4, but the model is still slow, almost as slow as the original FP32 ONNX model. Is it because of these warnings that I get? Or is it something else regarding the Deepstream config file?

We are moving this post to the Deepstream forum to get better help regarding the above.

Thank you.