Failed to parse onnx file

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU): Jetson Orin Nano
• DeepStream Version: deepstream-6.4
• JetPack Version (valid for Jetson only): Version: 6.0-b52
• TensorRT Version: Version: 8.6.2
• NVIDIA GPU Driver Version (valid for GPU only): NA
• Issue Type( questions, new requirements, bugs)

We are using mobilenet_v3_large for detection and developed the best_model.onnx file for it.

Then by using the following command, we developed the engine file

t-tech@ubuntu:~/OnnxToTRT$ trtexec --onnx=best_model.onnx --saveEngine=best_emodel_engine_pytorch.trt

These are the attached config files through which we are trying to run the custom model in the deepstream-app but getting errors (attached file as DD_error2.txt).

DD_Deepstream_basic.txt (1.5 KB)
DD_model_basic.txt (764 Bytes)
DD_error2.txt (4.0 KB)

• How to reproduce the issue? (This is for bugs. Including which sample app is used, the configuration files content, the command line used, and other details for reproducing)

Currently, we can not share this as the project is confidential.

• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Please help me to resolve the issue. Thanks.

Your deepstream is unable to load the trt engine file and thus is trying to create by itself where it fails.

Can you try loading the engine using trtexec first?

@rajupadhyay59 Thanks for your response.

Yes I tried to load the engine file trtexec It failed

[03/07/2024-14:08:13] [I] Note: The application clock rates do not reflect the actual clock rates that the GPU is currently running at.
[03/07/2024-14:08:13] [I] 
[03/07/2024-14:08:13] [I] TensorRT version: 8.6.2
[03/07/2024-14:08:13] [I] Loading standard plugins
[03/07/2024-14:08:13] [E] Error opening engine file: best_emodle_engine_pytorch.trt
[03/07/2024-14:08:13] [E] Failed to create engine from model or file.
[03/07/2024-14:08:13] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8602] # /usr/src/tensorrt/bin/trtexec --loadEngine=best_emodle_engine_pytorch.trt

What is wrong with the file?

So This time I tried to create another engine file using the following command but with .engine extension

t-tech@ubuntu:~/OnnxToTRT$ sudo /usr/src/tensorrt/bin/trtexec --onnx=best_model.onnx --saveEngine=best_model_engine_pytorch.engine

and when the same best_model_engine_pytorch.engine file get loaded successfully using the following command

t-tech@ubuntu:~/OnnxToTRT$ sudo /usr/src/tensorrt/bin/trtexec --loadEngine=best_model_engine_pytorch.engine

But still, the error remains the same with the new best_model_engine_pytorch.engine file .

You can keep .trt as an extension or .engine or anything else too.
It is basically a serialized engine which would be deserialized before inference begins. Extension has got nothing to do with it.

It is just that this time your model was built successfully.

Is there a reason you are using sudo while creating the engine?

What is the error this time?
Last time as I said before, your deepstream was simply not able to find the engine so it started building the engine using the onnx file and thus it failed.


Thanks for your response.

There is no specific reason for using sudo while creating the engine.

The error remains the same as I mentioned in the DD_error2.txt file when I created this post.

Try keeping the engine in the same directory where you config file is so that there is atleast no mistake in reading the while.

Also generate an engine without sudo.

Like I mentioned, if the error is same then your pipeline is simply not able to find your file so generate without sudo and keep in the same dir.

Okay, I create the best_model.engine engine file using the same command without using sudo.

t-tech@ubuntu:~/OnnxToTRT$ /usr/src/tensorrt/bin/trtexec --onnx=best_model.onnx --saveEngine=best_model.engine

which result in

&&&& PASSED TensorRT.trtexec [TensorRT v8602] # /usr/src/tensorrt/bin/trtexec --onnx=best_model.onnx --saveEngine=best_model.engine

Now even I can load the engine file. I replaced the engine file in the same directory where my configuration file is located.

Now the previous error vanished but got the following error

0:00:09.932346267  5053 0xaaab21664f60 ERROR                nvinfer gstnvinfer.cpp:676:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::parseBoundingBox() <nvdsinfer_context_impl_output_parsing.cpp:59> [UID = 1]: Could not find output coverage layer for parsing objects
0:00:09.932411484  5053 0xaaab21664f60 ERROR                nvinfer gstnvinfer.cpp:676:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::fillDetectionOutput() <nvdsinfer_context_impl_output_parsing.cpp:735> [UID = 1]: Failed to parse bboxes

My model output layer is output, which I repalce here output-blob-names=conv2d_bbox;conv2d_cov/Sigmoid
but still the same error?

Yes so most likely it was the path issue.

This error is pretty standard.

For every custom model, you need to write your own custom post-processing function.

You can search it on this forum and you’ll find some helpful links.

You can also search for yolo deepstream and you’ll find out how to make custom post-processing for custom models and you can just modify it for your own model.

Good luck.

@rajupadhyay59 Thanks for your help, things work for me.

I came to know that I need to add the following lines in my DD_model_basic.txt file under [property] to handle custom bounding box parsing for your model.


So I have added these lines but things do not work as this file .so and the corresponding function does not exist.

After reading I come to know about the following directory

t-tech@ubuntu:/opt/nvidia/deepstream/deepstream-6.4/sources/libs/nvdsinfer_customparser$ ls
Makefile  nvdsinfer_custombboxparser.cpp  nvdsinfer_customclassifierparser.cpp  nvdsinfer_customsegmentationparser.cpp  README

after reading README file I can create
in the directory.

t-tech@ubuntu:/opt/nvidia/deepstream/deepstream-6.4/sources/libs/nvdsinfer_customparser$ ls  Makefile  nvdsinfer_custombboxparser.cpp  nvdsinfer_customclassifierparser.cpp  nvdsinfer_customsegmentationparser.cpp  README

then add the following lines in the DD_model_basic.txt

# ... other properties ...

and things work. But I have some questions about how things work.

My model is mobilenet_v3_large which only does the classification task without a SSD, but how do network-type=1, and NvDsInferParseCustomTfSSD help to detect the bounding boxes?

network-mode=1 means that your engine was built using int8 precision.
You can go through tensorrt’s documentation for more information.

If you model is a classifier than you will use
parse-classifier-func-name instead of the parse-bbox-func-name.

NvDsInferParseCustomTfSSD does not help with the detection.
Your nvinfer element does the inference.

NvDsInferParseCustomTfSSD is basically a post-processing function which takes the results from nvinfer, and then we set the format like yolo follows center x, center y, width and height but detectron2 follows left, top, right, bottom.

You should refer to the Deepstream documentation for more info.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.