Orin AGX TensorRT conversion from torch.onnx.export fails

Description

A clear and concise description of the bug or issue.

I keep getting this error, no matter which model I’m converting
The bias tensor is required to be an initializer for the Conv operator.

I’m in the process of researching/selecting “toy” segmentation model architects by following the concepts in this guide https://github.com/NVIDIA-AI-IOT/jetson_dla_tutorial
Here’s the pseudo code that I used to convert pytorch models to onnx models
I’m using pretrained models from https://smp.readthedocs.io/en/latest/index.html and https://huggingface.co/docs/transformers/model_doc/segformer#segformer

Is there something I’m missing? How can I fix this ?

import torch
from torch_models import SegformerForSemanticSegmentation
import segmentation_models_pytorch as smp

"""
1) Instansiate a Model that needs to be exported
"""
models = {
	"model_segformer": SegformerForSemanticSegmentation.from_pretrained("nvidia/mit-b0", num_labels=14, id2label=id2label, label2id=label2id).eval(),
	"model_transformerunet": smp.Unet(encoder_name="mit_b0", encoder_weights="imagenet", in_channels=3, classes=14).eval(),
	"model_mobilenetv2unet": smp.Unet(encoder_name="mobilenet_v2", encoder_weights="imagenet", in_channels=3, classes=14).eval(),
	"model_mobilenetv3unet": smp.Unet(encoder_name="timm-mobilenetv3_large_100", encoder_weights="imagenet", in_channels=3, classes=14).eval(),
	"model_resnetunet": smp.Unet(encoder_name="resnet18", encoder_weights="imagenet", in_channels=3, classes=14).eval()
}

"""
2) Create a dummy input with the same shape as the real input
(batch, channels, height, width)
"""
data = torch.zeros(1, 3, 576, 960)

"""
3) Export torch model to onnx
"""
for name, model in models.items():
	torch.onnx.export(model, data, name+'.onnx', opset_version=11,
	    input_names=['input'],
	    output_names=['output'],
	    dynamic_axes={
	        'input': {0: 'batch_size'},
	        'output': {0: 'batch_size'}
	    }
	)

Here’s a sample of the command line that I use to convert the exported onnx to engine file

trtexec --onnx=model_segformer.onnx --shapes=input:1x3x576x960 --saveEngine=model_segformer.engine --exportProfile=model_segformer.json --int8 --useDLACore=0 --allowGPUFallback --useSpinWait --separateProfileRun > model_segformer.log

Environment

TensorRT Version: 8.4.1.5
GPU Type: ORIN AGX
Nvidia Driver Version:
CUDA Version: cuda_11.4.r11.4
CUDNN Version: 8.4.1.50
Operating System + Version: Jetpack 5.0.2
Python Version (if applicable): 3.8.10
TensorFlow Version (if applicable):
PyTorch Version (if applicable): 1.12.0a0+2c916ef.nv22.3
Baremetal or Container (if container which image + tag):

Relevant Files

Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)
model_mobilenetv2unet.onnx (25.2 MB)
model_mobilenetv3unet.onnx (25.5 MB)
model_resnetunet.onnx (54.7 MB)
model_segformer.onnx (14.3 MB)
model_transformerunet.onnx (21.3 MB)

Steps To Reproduce

Please include:

Hi,

We couldn’t reproduce this issue on the latest TensorRT NGC container 22.10 (v8.5), we could successfully build the engine.
We are moving this to the Jetson AGX Orin forum to get better help.

Thank you.

Hi,

Thanks for reporting this.

Confirmed that we can reproduce the error on an Orin + JetPack 5.0.2 environment.

$ /usr/src/tensorrt/bin/trtexec --onnx=model_segformer.onnx --shapes=input:1x3x576x960 --saveEngine=model_segformer.engine --exportProfile=model_segformer.json --int8 --useDLACore=0 --allowGPUFallback --useSpinWait --separateProfileRun > model_segformer.log
[11/16/2022-02:44:46] [W] [TRT] onnx2trt_utils.cpp:367: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[11/16/2022-02:44:51] [E] [TRT] ModelImporter.cpp:773: While parsing node number 1524 [Conv -> "input.312"]:
[11/16/2022-02:44:51] [E] [TRT] ModelImporter.cpp:774: --- Begin node ---
[11/16/2022-02:44:51] [E] [TRT] ModelImporter.cpp:775: input: "input.304"
input: "onnx::Conv_1865"
input: "onnx::Conv_1866"
output: "input.312"
name: "Conv_1524"
op_type: "Conv"
attribute {
  name: "dilations"
  ints: 1
  ints: 1
  type: INTS
}
attribute {
  name: "group"
  i: 1
  type: INT
}
attribute {
  name: "kernel_shape"
  ints: 1
  ints: 1
  type: INTS
}
attribute {
  name: "pads"
  ints: 0
  ints: 0
  ints: 0
  ints: 0
  type: INTS
}
attribute {
  name: "strides"
  ints: 1
  ints: 1
  type: INTS
}

[11/16/2022-02:44:51] [E] [TRT] ModelImporter.cpp:776: --- End node ---
[11/16/2022-02:44:51] [E] [TRT] ModelImporter.cpp:778: ERROR: ModelImporter.cpp:163 In function parseGraph:
[6] Invalid Node - Conv_1524
The bias tensor is required to be an initializer for the Conv operator. Try applying constant folding on the model using Polygraphy: https://github.com/NVIDIA/TensorRT/tree/master/tools/Polygraphy/examples/cli/surgeon/02_folding_constants
[11/16/2022-02:44:51] [E] Failed to parse onnx file
[11/16/2022-02:44:51] [E] Parsing model failed
[11/16/2022-02:44:52] [E] Failed to create engine from model or file.
[11/16/2022-02:44:52] [E] Engine set up failed

We are checking the details internally.
Will share more information with you later.

Thanks.

Hi,

The example uses ModelGN.

Do you also meet the error with the model?
Or this failure is specific to the segmentation model?

Thanks.

Using the example gave no errors.

Hi,

It looks like there are some non-supported layers in your segmentation model.

Since the model can work on TensorRT NGC container 22.10 (v8.5) (comment on Nov 10).
The support to handle the layer Conv_1524 should be added in TensorRT’s future release.

Please wait for our announcement for the next JetPack software.
Thanks.