Performance Drop After Converting YOLO ReLU6 Model from ONNX to DeepStream: Causes and Solutions

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hi,
After converting a YOLO ( classifcation) model from ONNX to DeepStream format, I noticed that the model’s performance decreased significantly, from an F1 score of 98% to 67%!
Why is this happening, and is there a method to correct the situation?

1 Like
2

When converting to DeepStream, how do you run the evaluation, and are the preprocessing steps between DeepStream and when you run PyTorch the same?

4

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)

• DeepStream Version

• JetPack Version (valid for Jetson only)

• TensorRT Version

• NVIDIA GPU Driver Version (valid for GPU only)

• Issue Type( questions, new requirements, bugs)

• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

5

One reason this may occur is due to Ultralytics using a padding with (114,114,114), while DeepStream uses (0,0,0).

6

Thank you very much for your response @Y-T-G . I appreciate your insight on the padding issue.

To resolve this problem and make the padding compatible between DeepStream and Ultralytics, do you suggest a specific approach or configuration to modify the padding values? Should I adjust the model preprocessing steps to ensure consistency, or would you recommend another solution?

Looking forward to your guidance.

Best regards

7

Thank you very much for your response @lqdisme. When I convert the model to DeepStream, I evaluate the performance by testing the model on the same test data (images) used to test the PyTorch model. I use the model generated with DeepStream and test this model with TensorRT.

8

I don’t think DeepStream supports padding with different color. The alternative is to modify Ultralytics to pad with 0,0,0

9

do you mean the TRT engine generated with DeepStream is used in PyTorch test?
please refer to this faq for “Debug Tips for DeepStream Accuracy Issue”.