Input length mismatch (onnx conversion to .trt)

Description

I have an .onnx model which I would like to convert to a .trt engine. In the beginning I got an error similar to the one described in this issue.

But after sanitisation of the model using the command:
./bin/polygraphy surgeon sanitize --fold-constants --no-shape-inference --save-external-data=data_small modelIn.onnx -o modelOut.onnx

I get a different error:

[07/08/2022-16:00:50] [TRT] [E] [shuffleNode.cpp::lowerParams::269] Error Code 4: Internal Error (Reshape_2573: second input must have length <= Dims::MAX_DIMS: length was 9)
[07/08/2022-16:00:50] [TRT] [E] ModelImporter.cpp:773: While parsing node number 708 [Reshape -> "onnx::Transpose_4140"]:
[07/08/2022-16:00:50] [TRT] [E] ModelImporter.cpp:774: --- Begin node ---
[07/08/2022-16:00:50] [TRT] [E] ModelImporter.cpp:775: input: "onnx::Reshape_4121"
input: "onnx::Reshape_4139"
output: "onnx::Transpose_4140"
name: "Reshape_2573"
op_type: "Reshape"

[07/08/2022-16:00:50] [TRT] [E] ModelImporter.cpp:776: --- End node ---
[07/08/2022-16:00:50] [TRT] [E] ModelImporter.cpp:779: ERROR: ModelImporter.cpp:180 In function parseGraph:
[6] Invalid Node - Reshape_2573
[shuffleNode.cpp::lowerParams::269] Error Code 4: Internal Error (Reshape_2573: second input must have length <= Dims::MAX_DIMS: length was 9)
[E] In node 708 (parseGraph): INVALID_NODE: Invalid Node - Reshape_2573
    [shuffleNode.cpp::lowerParams::269] Error Code 4: Internal Error (Reshape_2573: second input must have length <= Dims::MAX_DIMS: length was 9)
[!] Could not parse ONNX correctly

Environment

TensorRT Version: 8.4.1.5
GPU Type: Nvidia
Nvidia Driver Version:
CUDA Version: 11.5
CUDNN Version:
Operating System + Version: Ubuntu 20.04
Python Version (if applicable): 3.7
TensorFlow Version (if applicable):
PyTorch Version (if applicable): 1.10.1
Baremetal or Container (if container which image + tag):

Any ideas please?
Thank you,
Elias.

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.

In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

Excuse me, please ignore my previous response.

Here is my model:
model_small.onnx (66.1 MB)

Running trtexec without verbose produces:
/usr/src/tensorrt/bin/trtexec --onnx=model_small.onnx
no_verbose.txt (14.8 KB)

Running it with --verbose produces:
/usr/src/tensorrt/bin/trtexec --onnx=model_small.onnx --verbose
verbose.txt (1.5 MB)

After sanitising it, I get the following error:

./bin/polygraphy surgeon sanitize --fold-constants --no-shape-inference --save-external-data=model_small.onnx -o folded.onnx

[07/12/2022-10:52:56] [I] [TRT] MatMul_2528: broadcasting input1 to make tensors conform, dims(input0)=[1,2,3,3,4,4,4,64][NONE] dims(input1)=[1,1,1,1,1,1,64,192][NONE].
[07/12/2022-10:52:56] [I] [TRT] MatMul_2528: broadcasting input1 to make tensors conform, dims(input0)=[1,2,3,3,4,4,4,64][NONE] dims(input1)=[1,1,1,1,1,1,64,192][NONE].
[07/12/2022-10:52:56] [E] Error[4]: [shuffleNode.cpp::lowerParams::269] Error Code 4: Internal Error (Reshape_2548: second input must have length <= Dims::MAX_DIMS: length was 9)
[07/12/2022-10:52:56] [E] [TRT] ModelImporter.cpp:773: While parsing node number 708 [Reshape -> "onnx::Transpose_4128"]:
[07/12/2022-10:52:56] [E] [TRT] ModelImporter.cpp:774: --- Begin node ---
[07/12/2022-10:52:56] [E] [TRT] ModelImporter.cpp:775: input: "onnx::Reshape_4109"
input: "onnx::Reshape_4127"
output: "onnx::Transpose_4128"
name: "Reshape_2548"
op_type: "Reshape"

[07/12/2022-10:52:56] [E] [TRT] ModelImporter.cpp:776: --- End node ---
[07/12/2022-10:52:56] [E] [TRT] ModelImporter.cpp:779: ERROR: ModelImporter.cpp:180 In function parseGraph:
[6] Invalid Node - Reshape_2548
[shuffleNode.cpp::lowerParams::269] Error Code 4: Internal Error (Reshape_2548: second input must have length <= Dims::MAX_DIMS: length was 9)
[07/12/2022-10:52:56] [E] Failed to parse onnx file
[07/12/2022-10:52:57] [I] Finish parsing network model
[07/12/2022-10:52:57] [E] Parsing model failed
[07/12/2022-10:52:57] [E] Failed to create engine from model or file.
[07/12/2022-10:52:57] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8401] # /usr/src/tensorrt/bin/trtexec --onnx=folded.onnx

(this output is slightly different in the original message, as this was a slightly modified model…)

Thank you very much,
Elias.

Hi,

TensorRT doesn’t support 9-D tensors. We only support up to 8-D tensors.
You can work around this by modifying the original model so that it doesn’t contain any 9-D tensors.

Thank you.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.