Custom yolov8 model not working for int8

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) NVIDIA Geforce RTX 3090
• DeepStream Version7.1
• TensorRT Version10.3
**• NVIDIA GPU Driver Version (valid for GPU only)**560.35.03

I have implemented multistream deepstream pipeline using deepstream_test_3.py for custom yolov8 model with fp32 and it was working fine but when i am trying to run with network-mode=1 for int8 precision I am getting the following error.

File does not exist: /workspace/Primary_Detector/calib.table
WARNING: [TRT]: Calibration Profile is not defined. Calibrating with Profile 0
ERROR: [TRT]: [checkSanity.cpp::checkLinks::218] Error Code 2: Internal Error (Assertion item.second != nullptr failed. region should have been removed from Graph::regions)
/opt/nvidia/deepstream/deepstream-7.1/entrypoint.sh: line 15: 30 Segmentation fault (core dumped) /opt/nvidia/nvidia_entrypoint.sh $@

3

As The doc DeepStream-Yolo/docs/INT8Calibration.md at master · marcoslucianops/DeepStream-Yolo · GitHub suggests the calib.table file has to be generated provided the calibration data and calibration.txt along with the engine file but here due to some reason its not been generated. kindly help me to figure out what is going wrong.

4

you need to set an int8 calibration file if wanting to run model with network-mode=1(int8).
please refer to this topic for how to generate int8 calibration file.

5

I have followed the same topic to set up my calibration file . Kindly look into the actual error :

ERROR: [TRT]: [checkSanity.cpp::checkLinks::218] Error Code 2: Internal Error (Assertion item.second != nullptr failed. region should have been removed from Graph::regions)
/opt/nvidia/deepstream/deepstream-7.1/entrypoint.sh: line 15: 30 Segmentation fault (core dumped) /opt/nvidia/nvidia_entrypoint.sh $@

6

please refer to this topic. seems TensorRT10.4 can fix this issue.

7

removing version 10.3 and installing TensorRT10.4 worked for me. what can be the possible reason for it?

8

It is a bug of TensorRT 10.3. TensorRT 10.4 has fixed.

9

Can you please give detailed instructions on how to uninstall TensorRT 10.3 from JetPack 6.2 and install TensorRT 10.4 (on Jetson Orin Nano Developer Kit)? It would be of huge help.
Thanks in advance.

10

please refer to this link for how to install TensorRT. please refer to this link1, link2 for how to install TensorRT.