Please provide complete information as applicable to your setup.
**• Hardware Platform (Jetson / GPU) = Jetson nano
**• DeepStream Version = 6.0.1
**• JetPack Version (valid for Jetson only) = 4.6.1
**• TensorRT Version = 220.127.116.11-1+cuda10.2
**• Python version = 3.6.9
**• Issue Type( questions, new requirements, bugs) I am struggling to create a parser for the ONNX model that I have.
Hi, sorry for the long description, I wanted to make sure that I did not leave anything out.
I have created a pipeline that can receive a real time video stream, and then do object detection and object tracking on it. So far I have used the resnet10 model for inference, but I wanted to switch to a more accurate and up to date model. I am looking to use YOLOv8(and maybe later YoloNAS) as a inference model. I used this website to aid me in converting a yolov8.pt file to a ONNX file :
(In the website it says that there is a file called “gen_wts_yoloV8.py”, but I think that the script was renamed to “export_yoloV8.py”). The website also states that the function will extract the cfg- and weights files, where in fact it only exports the ONNX file (which is okay for my purpose). Just to make sure that the ONNX file was exported properly, I wrote a function that determines the outputs of the ONNX model, when I used this file to determine the outputs of the ONNX file, I got these results:
Output ‘boxes’ shape: [1, 8400, 4]
Output ‘scores’ shape: [1, 8400, 1]
Output ‘classes’ shape: [1, 8400, 1]
So it seems that the ONNX file was extracted properly.
(I used this command to create the ONNX file:
python3 export_yoloV8.py -w yolov8n.pt --dynamic
I had to use the --dynamic flag (instead of --simplify) because I was struggling to install the onnxsim package (I believe it is due to my python version not being up to date enough)
Now for the difficult part…
Deepstream only has these parsers prebuilt:
YoloV3 / YoloV3Tiny / YoloV2 / YoloV2Tiny
Here is the website where I found this information:
I am unsure of how to create a parser, so I looked online and found this website, to build the parser that can parse my ONNX file:
I used this command to build the .so file:
CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo
This is my config file for my inference model:
After I execute the program, I get this response:
Opening in BLOCKING MODE
Opening in BLOCKING MODE
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
gstnvtracker: Batch processing is ON
gstnvtracker: Past frame output is OFF
ERROR: Deserialize engine failed because file path: /opt/nvidia/deepstream/deepstream-6.0/sources/deepstream_python_apps/apps/deepstream-test1/model_b1_gpu0_fp32.engine open error
0:00:03.659054294 13578 0x15b26b50 WARN nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1889> [UID = 1]: deserialize engine from file :/opt/nvidia/deepstream/deepstream-6.0/sources/deepstream_python_apps/apps/deepstream-test1/model_b1_gpu0_fp32.engine failed
0:00:03.660209729 13578 0x15b26b50 WARN nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1996> [UID = 1]: deserialize backend context from engine from file :/opt/nvidia/deepstream/deepstream-6.0/sources/deepstream_python_apps/apps/deepstream-test1/model_b1_gpu0_fp32.engine failed, try rebuild
0:00:03.660261813 13578 0x15b26b50 INFO nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1914> [UID = 1]: Trying to create engine from model files
WARNING: [TRT]: onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
ERROR: [TRT]: [graph.cpp::computeInputExecutionUses::549] Error Code 9: Internal Error (Ceil_212: IUnaryLayer cannot be used to compute a shape tensor)
ERROR: [TRT]: ModelImporter.cpp:773: While parsing node number 214 [ConstantOfShape → “361”]:
ERROR: [TRT]: ModelImporter.cpp:774: — Begin node —
ERROR: [TRT]: ModelImporter.cpp:775: input: “360”
ERROR: [TRT]: ModelImporter.cpp:776: — End node —
ERROR: [TRT]: ModelImporter.cpp:779: ERROR: ModelImporter.cpp:179 In function parseGraph:
 Invalid Node - ConstantOfShape_214
[graph.cpp::computeInputExecutionUses::549] Error Code 9: Internal Error (Ceil_212: IUnaryLayer cannot be used to compute a shape tensor)
ERROR: Failed to parse onnx file
ERROR: failed to build network since parsing model errors.
ERROR: failed to build network.
0:00:04.299932545 13578 0x15b26b50 ERROR nvinfer gstnvinfer.cpp:632:gst_nvinfer_logger: NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1934> [UID = 1]: build engine file failed
0:00:04.301080479 13578 0x15b26b50 ERROR nvinfer gstnvinfer.cpp:632:gst_nvinfer_logger: NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2020> [UID = 1]: build backend context failed
0:00:04.301160324 13578 0x15b26b50 ERROR nvinfer gstnvinfer.cpp:632:gst_nvinfer_logger: NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1257> [UID = 1]: generate backend failed, check config file settings
0:00:04.301315014 13578 0x15b26b50 WARN nvinfer gstnvinfer.cpp:841:gst_nvinfer_start: error: Failed to create NvDsInferContext instance
0:00:04.301350588 13578 0x15b26b50 WARN nvinfer gstnvinfer.cpp:841:gst_nvinfer_start: error: Config file path: pgie_config_YOLO.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED
Error: gst-resource-error-quark: Failed to create NvDsInferContext instance (1): /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvinfer/gstnvinfer.cpp(841): gst_nvinfer_start (): /GstPipeline:pipeline0/GstNvInfer:primary-inference:
Config file path: pgie_config_YOLO.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED
I have looked around and could not find any resources that describes how to create a parser for a YoloV8 onnx file, and I am very unsure of what the actual problem might be. I’m assuming it is because the parser is faulty, but I could also be wrong. Does someone maybe know of anything that might send me in the right direction to solving this? Thank you very much for the effort.