Integrating a YOLOv11 ONNX Model into DeepStream: Requirements and Setup

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hello,

I trained a YOLOv11 model on a classification task and then exported the model in .onnx format using the following command:

path = model.export(format=“onnx”)

Now, I want to integrate the .onnx model into DeepStream.
How can I integrate this model?
Is adding the labels.txt and yolo11_relu6.onnx files sufficient for the model to work properly in DeepStream, or are there other files that need to be included?

3

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)

• DeepStream Version

• JetPack Version (valid for Jetson only)

• TensorRT Version

• NVIDIA GPU Driver Version (valid for GPU only)

• Issue Type( questions, new requirements, bugs)

• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

4 
       Driver Version: 535.183.01   CUDA Version: 12.2

Deepstream 6.3

5

classification labels.txt and onnx model are needed. please refer to nvinfer configuration sample for yolo. If using int8 accuracy, int8 calibration file is needed.

6

In my case, I would like the configuration for a YOLO classifier and not a detector!!

7 
from ultralytics import YOLO

# Load a model
model = YOLO("my_yolo11n-cls.pt")  # load an official model

# Export the model
model.export(format="onnx")

I used this code to export a YOLO model to the ONNX format, but I encountered an issue when I tried to use the ONNX model in DeepStream.
-----issue----------------------------------------------------------------------------------------
WARNING: [TRT]: onnx2trt_utils.cpp:374: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.

how to resolve this problem ?

8

I tried converting the my YOLO 11.onnx classifier model to .engine, but this warning appeared. Could this affect the model’s performance?

0:00:28.016518907 17743 0x7f49e4881c60 INFO nvinfer gstnvinfer.cpp:682:gst_nvinfer_logger:<nvinfer_bin_nvinfer> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:2002> [UID = 2]: Trying to create engine from model files
WARNING: [TRT]: onnx2trt_utils.cpp:374: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
WARNING: [TRT]: onnx2trt_utils.cpp:400: One or more weights outside the range of INT32 was clamped

9

@fanzh ?

10

It is because tensorrt does not support int64 weights. please ignore this TRT warning log “Attempting to cast down to INT32.”. Or you can convert the model to a new model without int64 weights.

11

How to convert INT64 weights to INT32 ?

12
  1. please refer to reply from TensorRT in this topic. I don’t think you’ll be losing precision unless the value is actually out of range of INT32. If that were the case, TensorRT would print another warning telling you that the cast was out of range.
  2. can your model work well in DeepStream now? if it still can’t work, please share the whole running log. please refer to this FAQ for How to use classification model as pgie?
13

Currently, the model is functional, but its performance decreases significantly in DeepStream. The YOLO 11 model consistently provides incorrect classifications, despite achieving a 96% F1 score outside of DeepStream during testing.

14

what do you mean about “96% F1 score”? does the accuracy drop significantly when testing in Deepstream? if it is a public model, could you share the model link? could you share the nvinfer configuration file? Thanks!

15

best.zip (5.2 MB)
config_YoloV11.txt (783 Bytes)

The accuracy drops significantly when testing in DeepStream.
Here is my custom YOLO 11 classification model with ReLU6 activation. Do you have any suggestions to improve the precision?

16
  1. what is the whole media pipeline? if FP16.onnx works as pgie, please set process-mode. please “try network-mode=0” first because using fp32 has the best accuracy. is the model created by DeepStream or trtexec?
  2. noticing the model works well when testing by other tool, please make sure the preprocessing configurations are the same. please find the parameter explanation in this doc.
17

In DeepStream, the model only displays class 1, even though I am using YOLO v11 as a classifier and not a detector. It never shows any class other than class 1 (I have 2 classes).

parser
parser1843×1034 90.4 KB

18

Is the output of a classification model in this form tensor: float32[1,2] (exemple [0.6, 0.4] ) compatible with DeepStream?
@fanzh

19

What is the required output format of a model for DeepStream when used as a classifier?

20

nvinfer plugin and low-level lib are opensource. the default postprocessing function is ClassifyPostprocessor::parseAttributesFromSoftmaxLayers. which can procss this kind of output
/* outputCoverageBuffer for classifiers is usually a softmax layer.
* The layer is an array of probabilities of the object belonging
* to each class with each probability being in the range [0,1] and
* sum all probabilities will be 1.
you also can add custom postprocessing function. please refer to this topic.