Issue with yolov8s-seg conversion from onnx to engine

revanthvarma2000 · October 28, 2024, 7:40pm

Facing the following issues while converting my yolov8 seg model from onnx to tensorrt :

/usr/src/tensorrt/bin/trtexec --onnx=yolov8s-seg.onnx --saveEngine=yolov8s-seg.engine
&&&& RUNNING TensorRT.trtexec [TensorRT v8401] # /usr/src/tensorrt/bin/trtexec --onnx=yolov8s-seg.onnx --saveEngine=yolov8s-seg.engine
[10/28/2024-19:31:33] [I] === Model Options ===
[10/28/2024-19:31:33] [I] Format: ONNX
[10/28/2024-19:31:33] [I] Model: yolov8s-seg.onnx
[10/28/2024-19:31:33] [I] Output:
[10/28/2024-19:31:33] [I] === Build Options ===
[10/28/2024-19:31:33] [I] Max batch: explicit batch
[10/28/2024-19:31:33] [I] Memory Pools: workspace: default, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
[10/28/2024-19:31:33] [I] minTiming: 1
[10/28/2024-19:31:33] [I] avgTiming: 8
[10/28/2024-19:31:33] [I] Precision: FP32
[10/28/2024-19:31:33] [I] LayerPrecisions:
[10/28/2024-19:31:33] [I] Calibration:
[10/28/2024-19:31:33] [I] Refit: Disabled
[10/28/2024-19:31:33] [I] Sparsity: Disabled
[10/28/2024-19:31:33] [I] Safe mode: Disabled
[10/28/2024-19:31:33] [I] DirectIO mode: Disabled
[10/28/2024-19:31:33] [I] Restricted mode: Disabled
[10/28/2024-19:31:33] [I] Build only: Disabled
[10/28/2024-19:31:33] [I] Save engine: yolov8s-seg.engine
[10/28/2024-19:31:33] [I] Load engine:
[10/28/2024-19:31:33] [I] Profiling verbosity: 0
[10/28/2024-19:31:33] [I] Tactic sources: Using default tactic sources
[10/28/2024-19:31:33] [I] timingCacheMode: local
[10/28/2024-19:31:33] [I] timingCacheFile:
[10/28/2024-19:31:33] [I] Input(s)s format: fp32:CHW
[10/28/2024-19:31:33] [I] Output(s)s format: fp32:CHW
[10/28/2024-19:31:33] [I] Input build shapes: model
[10/28/2024-19:31:33] [I] Input calibration shapes: model
[10/28/2024-19:31:33] [I] === System Options ===
[10/28/2024-19:31:33] [I] Device: 0
[10/28/2024-19:31:33] [I] DLACore:
[10/28/2024-19:31:33] [I] Plugins:
[10/28/2024-19:31:33] [I] === Inference Options ===
[10/28/2024-19:31:33] [I] Batch: Explicit
[10/28/2024-19:31:33] [I] Input inference shapes: model
[10/28/2024-19:31:33] [I] Iterations: 10
[10/28/2024-19:31:33] [I] Duration: 3s (+ 200ms warm up)
[10/28/2024-19:31:33] [I] Sleep time: 0ms
[10/28/2024-19:31:33] [I] Idle time: 0ms
[10/28/2024-19:31:33] [I] Streams: 1
[10/28/2024-19:31:33] [I] ExposeDMA: Disabled
[10/28/2024-19:31:33] [I] Data transfers: Enabled
[10/28/2024-19:31:33] [I] Spin-wait: Disabled
[10/28/2024-19:31:33] [I] Multithreading: Disabled
[10/28/2024-19:31:33] [I] CUDA Graph: Disabled
[10/28/2024-19:31:33] [I] Separate profiling: Disabled
[10/28/2024-19:31:33] [I] Time Deserialize: Disabled
[10/28/2024-19:31:33] [I] Time Refit: Disabled
[10/28/2024-19:31:33] [I] Inputs:
[10/28/2024-19:31:33] [I] === Reporting Options ===
[10/28/2024-19:31:33] [I] Verbose: Disabled
[10/28/2024-19:31:33] [I] Averages: 10 inferences
[10/28/2024-19:31:33] [I] Percentile: 99
[10/28/2024-19:31:33] [I] Dump refittable layers:Disabled
[10/28/2024-19:31:33] [I] Dump output: Disabled
[10/28/2024-19:31:33] [I] Profile: Disabled
[10/28/2024-19:31:33] [I] Export timing to JSON file:
[10/28/2024-19:31:33] [I] Export output to JSON file:
[10/28/2024-19:31:33] [I] Export profile to JSON file:
[10/28/2024-19:31:33] [I]
[10/28/2024-19:31:33] [I] === Device Information ===
[10/28/2024-19:31:33] [I] Selected Device: Orin
[10/28/2024-19:31:33] [I] Compute Capability: 8.7
[10/28/2024-19:31:33] [I] SMs: 8
[10/28/2024-19:31:33] [I] Compute Clock Rate: 1.3 GHz
[10/28/2024-19:31:33] [I] Device Global Memory: 30535 MiB
[10/28/2024-19:31:33] [I] Shared Memory per SM: 164 KiB
[10/28/2024-19:31:33] [I] Memory Bus Width: 128 bits (ECC disabled)
[10/28/2024-19:31:33] [I] Memory Clock Rate: 1.3 GHz
[10/28/2024-19:31:33] [I]
[10/28/2024-19:31:33] [I] TensorRT version: 8.4.1
[10/28/2024-19:31:34] [I] [TRT] [MemUsageChange] Init CUDA: CPU +218, GPU +0, now: CPU 242, GPU 7604 (MiB)
[10/28/2024-19:31:38] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +351, GPU +331, now: CPU 612, GPU 7953 (MiB)
[10/28/2024-19:31:38] [I] Start parsing network model
[10/28/2024-19:31:38] [I] [TRT] ----------------------------------------------------------------
[10/28/2024-19:31:38] [I] [TRT] Input filename: yolov8s-seg.onnx
[10/28/2024-19:31:38] [I] [TRT] ONNX IR version: 0.0.8
[10/28/2024-19:31:38] [I] [TRT] Opset version: 16
[10/28/2024-19:31:38] [I] [TRT] Producer name: pytorch
[10/28/2024-19:31:38] [I] [TRT] Producer version: 1.13.0
[10/28/2024-19:31:38] [I] [TRT] Domain:
[10/28/2024-19:31:38] [I] [TRT] Model version: 0
[10/28/2024-19:31:38] [I] [TRT] Doc string:
[10/28/2024-19:31:38] [I] [TRT] ----------------------------------------------------------------
[10/28/2024-19:31:38] [W] [TRT] onnx2trt_utils.cpp:367: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[10/28/2024-19:31:39] [W] [TRT] Tensor DataType is determined at build time for tensors not marked as input or output.
[10/28/2024-19:31:39] [W] [TRT] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[10/28/2024-19:31:39] [I] [TRT] No importer registered for op: RoiAlign. Attempting to import as plugin.
[10/28/2024-19:31:39] [I] [TRT] Searching for plugin: RoiAlign, plugin_version: 1, plugin_namespace:
[10/28/2024-19:31:39] [E] [TRT] ModelImporter.cpp:773: While parsing node number 608 [RoiAlign → “onnx::Cast_815”]:
[10/28/2024-19:31:39] [E] [TRT] ModelImporter.cpp:774: — Begin node —
[10/28/2024-19:31:39] [E] [TRT] ModelImporter.cpp:775: input: “onnx::Shape_354”
input: “selected_boxes”
input: “onnx::Mul_752”
output: “onnx::Cast_815”
name: “RoiAlign_608”
op_type: “RoiAlign”
attribute {
name: “coordinate_transformation_mode”
s: “half_pixel”
type: STRING
}
attribute {
name: “mode”
s: “avg”
type: STRING
}
attribute {
name: “output_height”
i: 160
type: INT
}
attribute {
name: “output_width”
i: 160
type: INT
}
attribute {
name: “sampling_ratio”
i: 0
type: INT
}
attribute {
name: “spatial_scale”
f: 0.25
type: FLOAT
}

[10/28/2024-19:31:39] [E] [TRT] ModelImporter.cpp:776: — End node —
[10/28/2024-19:31:39] [E] [TRT] ModelImporter.cpp:778: ERROR: builtin_op_importers.cpp:4890 In function importFallbackPluginImporter:
[8] Assertion failed: creator && “Plugin not found, are the plugin name, version, and namespace correct?”
[10/28/2024-19:31:39] [E] Failed to parse onnx file
[10/28/2024-19:31:39] [I] Finish parsing network model
[10/28/2024-19:31:39] [E] Parsing model failed
[10/28/2024-19:31:39] [E] Failed to create engine from model or file.
[10/28/2024-19:31:39] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8401] # /usr/src/tensorrt/bin/trtexec --onnx=yolov8s-seg.onnx --saveEngine=yolov8s-seg.engine

I am using jetson orin, with
Jetpack : 5.0.2
Cuda : 11.4
tensorrt : 8.4.1.5
onnx: 1.12.0
onnxsim : 0.4.36

carolyuu · October 28, 2024, 8:00pm

Hi,
We support jetson_multimedia_api and gstreamer. Please make sure you have checked the document:
https://docs.nvidia.com/jetson/archives/r36.3/DeveloperGuide/SD/Multimedia.html
https://docs.nvidia.com/jetson/archives/r36.3/ApiReference/index.html
There are some examples in

Q: Is there any example of running RTSP streaming?
Q: Is there an example for running UDP streaming?
Q: I have a USB camera. How can I launch it on AGX Orin?

For further issues, please share a method to replicate the issue through gstreamer command, or either sample. We will set up developer kit and check.

Thanks!

revanthvarma2000 · October 28, 2024, 8:12pm

Hi, seems your answer is different from what I asked. Could you please look into this again ?

revanthvarma2000 · October 28, 2024, 8:14pm

My issue is similar to this : TRT Fails to parse ONNX Model (yolov8 Segmentation)

But in my case, I am already using the latest version mentioned in the solved post

AastaLLL · October 29, 2024, 2:39am

Hi,

[10/28/2024-19:31:39] [E] [TRT] ModelImporter.cpp:775: input: “onnx::Shape_354”
...
op_type: “RoiAlign”
...

The layer is not supported in TensorRT 8.4.
You can find the supported matrix below:

github.com

onnx/onnx-tensorrt/blob/8.4-GA/docs/operators.md

<!--- SPDX-License-Identifier: Apache-2.0 -->

# Supported ONNX Operators

TensorRT 8.4 supports operators up to Opset 17. Latest information of ONNX operators can be found [here](https://github.com/onnx/onnx/blob/master/docs/Operators.md)

TensorRT supports the following ONNX data types: DOUBLE, FLOAT32, FLOAT16, INT8, and BOOL

> Note: There is limited support for INT32, INT64, and DOUBLE types. TensorRT will attempt to cast down INT64 to INT32 and DOUBLE down to FLOAT, clamping values to `+-INT_MAX` or `+-FLT_MAX` if necessary.

See below for the support matrix of ONNX operators in ONNX-TensorRT.

## Operator Support Matrix

| Operator                  | Supported  | Supported Types | Restrictions                                                                                                           |
|---------------------------|------------|-----------------|------------------------------------------------------------------------------------------------------------------------|
| Abs                       | Y          | FP32, FP16, INT32 |
| Acos                      | Y          | FP32, FP16 |
| Acosh                     | Y          | FP32, FP16 |
| Add                       | Y          | FP32, FP16, INT32 |

This file has been truncated. show original

Please try the model with a newer TensorRT version as it is added after TensoerRT 8.5.

Thanks.

system · November 20, 2024, 5:31am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
TensorRT problem on NVIDIA APEX ORIN NX TensorRT tensorrt , jetson-inference , cudnn	1	23	August 29, 2024
TensorRT quantization bug on Jetpack 6.0 Jetson AGX Orin tensorrt , pytorch	6	561	January 22, 2024
Keras->Onnx->TensorRT Jetson AGX Orin tensorrt	4	54	September 25, 2024
I do not get any performance improvement after using TensorRT provider for object detection model Jetson Nano tensorrt , onnx	7	1380	July 12, 2022
Running a pytorch network converted to ONNX with TensorRT on the TX2 Jetson TX2	24	8795	October 18, 2021
Process killed during tensorrt conversion on Jetson orin NX (8 GB) Jetson Orin NX tensorrt	15	664	April 30, 2024
ERORR with ONNX2TRT : Unknown embedded device detected Jetson Xavier NX onnx	18	4480	April 27, 2022
How to deploy attention on deepstream6.0? DeepStream SDK	3	171	December 5, 2023
Erorr with onnx to trt Jetson Xavier NX tensorrt	8	1230	March 30, 2022
Jetson Nano Python 3.7 version for Tensorrt Jetson Nano tensorrt , python	14	3698	April 12, 2023

Issue with yolov8s-seg conversion from onnx to engine

Related topics