Failed to convert the SAM2.1 decoder onnx model due to Error Code 4: Internal Error

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Greetings, everyone.
Here is an onnx file: https://github.com/ibaiGorordo/ONNX-SAM2-Segment-Anything/releases/download/0.2.0/decoder.onnx

I am trying to use TensorRT to build convert this file using this cli:
trtexec --onnx=decoder.onnx --saveEngine=decoder.engine

hardware information output by TensorRT :
[04/20/2025-17:31:16] [I] === Device Information ===
[04/20/2025-17:31:16] [I] Available Devices:
[04/20/2025-17:31:16] [I] Device 0: “Orin” UUID: GPU-a00bb704-da56-555b-a79c-65a4e3662de8
[04/20/2025-17:31:16] [I] Selected Device: Orin
[04/20/2025-17:31:16] [I] Selected Device ID: 0
[04/20/2025-17:31:16] [I] Selected Device UUID: GPU-a00bb704-da56-555b-a79c-65a4e3662de8
[04/20/2025-17:31:16] [I] Compute Capability: 8.7
[04/20/2025-17:31:16] [I] SMs: 16
[04/20/2025-17:31:16] [I] Device Global Memory: 62840 MiB
[04/20/2025-17:31:16] [I] Shared Memory per SM: 164 KiB
[04/20/2025-17:31:16] [I] Memory Bus Width: 256 bits (ECC disabled)
[04/20/2025-17:31:16] [I] Application Compute Clock Rate: 1.3 GHz
[04/20/2025-17:31:16] [I] Application Memory Clock Rate: 1.3 GHz

TensorRT version is shown here[The cuda version is 12.6]:
[04/20/2025-17:31:16] [I] Note: The application clock rates do not reflect the actual clock rates that the GPU is currently running at.
[04/20/2025-17:31:16] [I] TensorRT version: 10.7.0
[04/20/2025-17:31:16] [I] Loading standard plugins
[04/20/2025-17:31:16] [I] [TRT] [MemUsageChange] Init CUDA: CPU +2, GPU +0, now: CPU 31, GPU 13403 (MiB)
[04/20/2025-17:31:18] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +928, GPU +749, now: CPU 1002, GPU 14197 (MiB)
[04/20/2025-17:31:18] [I] Start parsing network model.
[04/20/2025-17:31:18] [I] [TRT] ----------------------------------------------------------------
[04/20/2025-17:31:18] [I] [TRT] Input filename: /home/nvidia/projects/segment-anything-2/decoder.onnx
[04/20/2025-17:31:18] [I] [TRT] ONNX IR version: 0.0.8
[04/20/2025-17:31:18] [I] [TRT] Opset version: 16
[04/20/2025-17:31:18] [I] [TRT] Producer name: pytorch
[04/20/2025-17:31:18] [I] [TRT] Producer version: 2.6.0
[04/20/2025-17:31:18] [I] [TRT] Domain:
[04/20/2025-17:31:18] [I] [TRT] Model version: 0
[04/20/2025-17:31:18] [I] [TRT] Doc string:
[04/20/2025-17:31:18] [I] [TRT] ----------------------------------------------------------------
[04/20/2025-17:31:18] [E] Error[4]: ITensor::getDimensions: Error Code 4: Internal Error (/OneHot: an IIOneHotLayer cannot be used to compute a shape tensor)
[04/20/2025-17:31:18] [E] [TRT] ModelImporter.cpp:948: While parsing node number 146 [Tile → “/Tile_output_0”]:
[04/20/2025-17:31:18] [E] [TRT] ModelImporter.cpp:950: — Begin node —
input: “/Unsqueeze_11_output_0”
input: “/Reshape_4_output_0”
output: “/Tile_output_0”
name: “/Tile”
op_type: “Tile”

[04/20/2025-17:31:18] [E] [TRT] ModelImporter.cpp:951: — End node —
[04/20/2025-17:31:18] [E] [TRT] ModelImporter.cpp:953: ERROR: ModelImporter.cpp:195 In function parseNode:
[6] Invalid Node - /Tile
ITensor::getDimensions: Error Code 4: Internal Error (/OneHot: an IIOneHotLayer cannot be used to compute a shape tensor)
[04/20/2025-17:31:18] [E] Failed to parse onnx file
[04/20/2025-17:31:18] [I] Finished parsing network model. Parse time: 0.037032
[04/20/2025-17:31:18] [E] Parsing model failed
[04/20/2025-17:31:18] [E] Failed to create engine from model or file.
[04/20/2025-17:31:18] [E] Engine set up failed

Is anyone willing to share any idea about this?
Thank you, kind community. :)

3

Hi @noname.mark09 ,
Can you pls try with min, max and opt shapes with trtexec command , and confirm back if you are still facing this issue?
Alternatively

  1. Review Layer Support:
  • The OneHot layer and related operations sometimes cause compatibility issues in TensorRT. According to a GitHub issue, an IIOneHotLayer cannot compute a shape tensor, leading to the ITensor::getDimensions error. This indicates that TensorRT might not fully support the OneHot operation as implemented in your ONNX model.
  • Solution: Consider replacing or simplifying this part of the model before the conversion, possibly by using a custom implementation or alternative logic for one-hot encoding.
  1. Verify ONNX Model Version and Operations:
  • Ensure your ONNX model uses supported operations for the TensorRT version you are utilizing. TensorRT tends to lag in support for the latest ONNX features or operations.
  • Solution: Check the opset version of your ONNX model. It might be beneficial to downgrade to an earlier opset version that is known to work smoothly with TensorRT.
  1. Use Latest Versions:
  • Make sure you are using the latest version of both TensorRT and the ONNX-TensorRT parser, as improvements and bug fixes could resolve your issue.
  • Solution: Update TensorRT and ensure you are using the appropriate version of CUDA and cuDNN compatible with your TensorRT installation.
  1. Inspect Input Types:
  • Ensure that the input types for your layers are supported and correctly formatted. As per logs related to similar errors, TensorRT doesn’t natively support certain data types like UINT8.
  • Solution: Cast all relevant inputs to supported types like INT32 or FLOAT32 prior to conversion.
  1. Alter the Model:
  • If the error continues, consider modifying the original ONNX model before exporting it. Using libraries like PyTorch or TensorFlow to manually alter the operations could help create a more compatible model for TensorRT.
  • Example: You might replace or remove layers that trigger incompatibility issues or reconfigure how data flows through certain layers.
  1. Using Workaround Techniques:
  • Some users have reported success by exporting their models with alternate flags or utilizing additional parameters in the trtexec command.
  • Solution: Try adding the --explicitBatch flag or modifying how inputs are defined during conversion.
  1. Debugging Tools:
  • Utilize TensorRT’s debugging facilities to gain insight into where the conversion process is failing. This might involve checking the verbose output logs during the conversion process.
  • Solution: Use trtexec --onnx=model.onnx --verbose to get detailed logs that could highlight the exact failure point.

Thanks