TRT Fails to parse ONNX Model (yolov8 Segmentation)

gysi · February 9, 2024, 8:17am

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU): Jetson Nvidia Xavier NX
• DeepStream Version: 6.0
• JetPack Version (valid for Jetson only)
• TensorRT Version: 8.2.1.8
• NVIDIA GPU Driver Version (valid for GPU only):
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Hi

I tried to serialize a yolov8-Segmentation Network but it fails.

I converted the YoloV8 Model using the instructions given from GitHub - marcoslucianops/DeepStream-Yolo-Seg: NVIDIA DeepStream SDK 6.3 / 6.2 / 6.1.1 / 6.1 / 6.0.1 / 6.0 implementation for YOLO-Segmentation models .
When trying to generate the TRT it fails with the following error.

I also like to mention that I used many yoloV8 models for object detection and I never had an issue while serializing.
The issue only occurs when using segmentation model.

nvidia@ubuntu:/etc/stereo_camera/dnn/DeepStream-Yolo-Seg$ deepstream-app -c deepstream_app_config.txt 

Using winsys: x11 
ERROR: Deserialize engine failed because file path: /etc/villiger/gripper/seg_dnn/yolov8s-seg.onnx_b1_gpu0_fp32.engine open error
0:00:03.498454304  7679   0x7f300022c0 WARN                 nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1889> [UID = 1]: deserialize engine from file :/etc/villiger/gripper/seg_dnn/yolov8s-seg.onnx_b1_gpu0_fp32.engine failed
0:00:03.521018368  7679   0x7f300022c0 WARN                 nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1996> [UID = 1]: deserialize backend context from engine from file :/etc/villiger/gripper/seg_dnn/yolov8s-seg.onnx_b1_gpu0_fp32.engine failed, try rebuild
0:00:03.521485568  7679   0x7f300022c0 INFO                 nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1914> [UID = 1]: Trying to create engine from model files
WARNING: [TRT]: onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
WARNING: [TRT]: Tensor DataType is determined at build time for tensors not marked as input or output.
WARNING: [TRT]: onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
ERROR: [TRT]: ModelImporter.cpp:773: While parsing node number 449 [RoiAlign -> "/1/RoiAlign_output_0"]:
ERROR: [TRT]: ModelImporter.cpp:774: --- Begin node ---
ERROR: [TRT]: ModelImporter.cpp:775: input: "/0/model.22/proto/cv3/act/Mul_output_0"
input: "/1/Reshape_1_output_0"
input: "/1/Gather_1_output_0"
output: "/1/RoiAlign_output_0"
name: "/1/RoiAlign"
op_type: "RoiAlign"
attribute {
  name: "coordinate_transformation_mode"
  s: "half_pixel"
  type: STRING
}
attribute {
  name: "mode"
  s: "avg"
  type: STRING
}
attribute {
  name: "output_height"
  i: 320
  type: INT
}
attribute {
  name: "output_width"
  i: 320
  type: INT
}
attribute {
  name: "sampling_ratio"
  i: 0
  type: INT
}
attribute {
  name: "spatial_scale"
  f: 0.25
  type: FLOAT
}

ERROR: [TRT]: ModelImporter.cpp:776: --- End node ---
ERROR: [TRT]: ModelImporter.cpp:779: ERROR: builtin_op_importers.cpp:4870 In function importFallbackPluginImporter:
[8] Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"
ERROR: Failed to parse onnx file
ERROR: failed to build network since parsing model errors.
ERROR: failed to build network.
0:00:04.630773120  7679   0x7f300022c0 ERROR                nvinfer gstnvinfer.cpp:632:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1934> [UID = 1]: build engine file failed
0:00:04.656896928  7679   0x7f300022c0 ERROR                nvinfer gstnvinfer.cpp:632:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2020> [UID = 1]: build backend context failed
0:00:04.657157920  7679   0x7f300022c0 ERROR                nvinfer gstnvinfer.cpp:632:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1257> [UID = 1]: generate backend failed, check config file settings
0:00:04.657713088  7679   0x7f300022c0 WARN                 nvinfer gstnvinfer.cpp:841:gst_nvinfer_start:<primary_gie> error: Failed to create NvDsInferContext instance
0:00:04.657768064  7679   0x7f300022c0 WARN                 nvinfer gstnvinfer.cpp:841:gst_nvinfer_start:<primary_gie> error: Config file path: /etc/stereo_camera/dnn/DeepStream-Yolo-Seg/config_infer_primary_yoloV8_seg.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED
** ERROR: <main:707>: Failed to set pipeline to PAUSED
Quitting
ERROR from primary_gie: Failed to create NvDsInferContext instance
Debug info: /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvinfer/gstnvinfer.cpp(841): gst_nvinfer_start (): /GstPipeline:pipeline/GstBin:primary_gie_bin/GstNvInfer:primary_gie:
Config file path: /etc/stereo_camera/dnn/DeepStream-Yolo-Seg/config_infer_primary_yoloV8_seg.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED
App run failed

gysi · February 9, 2024, 9:24am

I also tried running the model with trtexec command.

This also fails with the following error:

[09/28/2023-16:24:31] [I] === Device Information ===
[09/28/2023-16:24:31] [I] Selected Device: Xavier
[09/28/2023-16:24:31] [I] Compute Capability: 7.2
[09/28/2023-16:24:31] [I] SMs: 6
[09/28/2023-16:24:31] [I] Compute Clock Rate: 1.109 GHz
[09/28/2023-16:24:31] [I] Device Global Memory: 7773 MiB
[09/28/2023-16:24:31] [I] Shared Memory per SM: 96 KiB
[09/28/2023-16:24:31] [I] Memory Bus Width: 256 bits (ECC disabled)
[09/28/2023-16:24:31] [I] Memory Clock Rate: 1.109 GHz
[09/28/2023-16:24:31] [I] 
[09/28/2023-16:24:31] [I] TensorRT version: 8.2.1
[09/28/2023-16:24:33] [I] [TRT] [MemUsageChange] Init CUDA: CPU +363, GPU +0, now: CPU 381, GPU 3622 (MiB)
[09/28/2023-16:24:34] [I] [TRT] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 381 MiB, GPU 3622 MiB
[09/28/2023-16:24:34] [I] [TRT] [MemUsageSnapshot] End constructing builder kernel library: CPU 486 MiB, GPU 3727 MiB
[09/28/2023-16:24:34] [I] Start parsing network model
[09/28/2023-16:24:35] [I] [TRT] ----------------------------------------------------------------
[09/28/2023-16:24:35] [I] [TRT] Input filename:   /etc/villiger/gripper/seg_dnn/yolov8s-seg.onnx
[09/28/2023-16:24:35] [I] [TRT] ONNX IR version:  0.0.8
[09/28/2023-16:24:35] [I] [TRT] Opset version:    16
[09/28/2023-16:24:35] [I] [TRT] Producer name:    pytorch
[09/28/2023-16:24:35] [I] [TRT] Producer version: 2.1.2
[09/28/2023-16:24:35] [I] [TRT] Domain:           
[09/28/2023-16:24:35] [I] [TRT] Model version:    0
[09/28/2023-16:24:35] [I] [TRT] Doc string:       
[09/28/2023-16:24:35] [I] [TRT] ----------------------------------------------------------------
[09/28/2023-16:24:35] [W] [TRT] onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[09/28/2023-16:24:35] [W] [TRT] Tensor DataType is determined at build time for tensors not marked as input or output.
[09/28/2023-16:24:35] [W] [TRT] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[09/28/2023-16:24:35] [I] [TRT] No importer registered for op: RoiAlign. Attempting to import as plugin.
[09/28/2023-16:24:35] [I] [TRT] Searching for plugin: RoiAlign, plugin_version: 1, plugin_namespace: 
[09/28/2023-16:24:35] [E] [TRT] ModelImporter.cpp:773: While parsing node number 449 [RoiAlign -> "/1/RoiAlign_output_0"]:
[09/28/2023-16:24:35] [E] [TRT] ModelImporter.cpp:774: --- Begin node ---
[09/28/2023-16:24:35] [E] [TRT] ModelImporter.cpp:775: input: "/0/model.22/proto/cv3/act/Mul_output_0"
input: "/1/Reshape_1_output_0"
input: "/1/Gather_1_output_0"
output: "/1/RoiAlign_output_0"
name: "/1/RoiAlign"
op_type: "RoiAlign"
attribute {
  name: "coordinate_transformation_mode"
  s: "half_pixel"
  type: STRING
}
attribute {
  name: "mode"
  s: "avg"
  type: STRING
}
attribute {
  name: "output_height"
  i: 320
  type: INT
}
attribute {
  name: "output_width"
  i: 320
  type: INT
}
attribute {
  name: "sampling_ratio"
  i: 0
  type: INT
}
attribute {
  name: "spatial_scale"
  f: 0.25
  type: FLOAT
}

[09/28/2023-16:24:35] [E] [TRT] ModelImporter.cpp:776: --- End node ---
[09/28/2023-16:24:35] [E] [TRT] ModelImporter.cpp:779: ERROR: builtin_op_importers.cpp:4870 In function importFallbackPluginImporter:
[8] Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"
[09/28/2023-16:24:35] [E] Failed to parse onnx file
[09/28/2023-16:24:35] [I] Finish parsing network model
[09/28/2023-16:24:35] [E] Parsing model failed
[09/28/2023-16:24:35] [E] Failed to create engine from model.
[09/28/2023-16:24:35] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8201] # ./trtexec --onnx=/etc/villiger/gripper/seg_dnn/yolov8s-seg.onnx

yuweiw · February 19, 2024, 7:51am

Could you share your onnx model to us?

gysi · February 19, 2024, 8:52am

Sure thing.
I used the basic model from ultralytics and converted it to onnx using the instructions provided by DeepStream-Yolo-Seg/docs/YOLOv8_Seg.md at master · marcoslucianops/DeepStream-Yolo-Seg · GitHub.

Basic model from ultralytics:
https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8s-seg.pt

ONNX Model:
yolov8s-seg.zip (38.2 MB)

AastaLLL · February 23, 2024, 9:17am

Hi,

TensorRT 8.2 has been released for a while.
Would you mind giving JetPack 5 a try?

Thanks.

system · March 27, 2024, 2:18am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
TRT Fails to parse ONNX Model (yolov8 Segmentation) DeepStream SDK tensorrt , yolo , onnx	4	293	June 11, 2024
ONNX as inference model (YoloV8) DeepStream SDK jetpack , onnx , deepstream	12	3432	July 10, 2023
ERR with YOLOv4 to TensoRT DeepStream SDK tensorrt	8	1472	October 12, 2021
Tutorial: How to run YOLOv7 on Deepstream DeepStream SDK demos-and-tutorials	19	6875	March 26, 2024
Generate engine using onnx2trt, The engine was used to call deepstream, but an error was reported DeepStream SDK onnx	5	217	June 11, 2024
TensorRT fails to parse bounding boxes and ONNX explicit batch DeepStream SDK tensorrt , deepstream	15	1381	April 20, 2023
Yolov7 onnx model convert to trt model error in deepstream 6.2, but use trtexec ok DeepStream SDK	9	332	June 11, 2024
Yolov8seg giving divide by 0 errors if no detection in frame DeepStream SDK	11	749	November 7, 2023
TAO Toolkit 4.0, Yolo v4 not working with Deepstream nvinfer - parseModel: Failed to parse ONNX model TAO Toolkit onnx , tao , deepstream61	6	964	February 20, 2023
Iplugin tensorrt engine error for ds5.0 DeepStream SDK	29	4151	October 12, 2021

TRT Fails to parse ONNX Model (yolov8 Segmentation)

Related topics