YOLOv3 TensorRT model in DeepStream

Accelerated Computing Intelligent Video Analytics DeepStream SDK
8

Hi Dilip.s
Confirmed that the DS(DeepStream) YoloV3 and TRT yolov3-onnx sample are based on the exactly same network and model (https://raw.githubusercontent.com/pjreddie/darknet/master/cfg/yolov3.cfg and https://pjreddie.com/media/files/yolov3.weights), but DS parses the yolov3.cfg and does some modification.

You can use below two commands to profile the TRT engine generated by them, you can find they have differnet layer number, and the last layer are different (DS YOLOV3 has a plugin to help post-precessing, TRT YOLOV3 use python tool to handle it).

  1. DeepStream yolov3:
    $ tensorrt/bin/trtexec --loadEngine=model_b4_fp32.engine --dumpProfile --plugins=/opt/nvidia/deepstream/deepstream-4.0/sources/objectDetector_Yolo/nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so
  2. TRT yolov3
    $ ~/tensorrt/bin/trtexec --loadEngine=model_b1_fp32.engine --dumpProfile

I didn’t look into the detailed difference, but I believe the real models running in DS and TRT YoloV3 are different. These difference and the post-processing causes the issue you saw above.

And, as you can find under /opt/nvidia/deepstream/deepstream-4.0/sources/objectDetector_Yolo/, there is yolov3-calibration.table.trt5.1, which is the INT8 calibration table for DS to run its Yolov3 under INT8 mode.

For topic - (Batch Size Failure in Custom YOLOv3 INT8), my colleage can’t reproduce the issue you mentioned, could you provide more details about the repo.

Thanks!

1 Like