Building RNN Layer failed in Jetson NX

Hi,

I have successfully built the TRT model on X86 but while building the TRT model on ARM64 (NX) failed. With following error.

GRU_75: inputs to IRecurrenceLayer mismatched
Builder failed while analyzing shapes.

X86 Config:
TensorRT version: 7.2.3.4

ARM64 config:
TensorRt version: 7.1.3.4
Installed through Jet Pack 4.5

I could not install newer version of TensorRT in ARM64. Could anyone help me here?

Hi,

Do you use ONNX model format?
If yes, could you share the model with us so we can give it a quick check?

Thanks.

model.onnx (1.7 MB)

Yes I use ONNX model format with opset_version=11

I have uploaded the ONNX model file for your reference.

Any update?

Hi,

Sorry for the late update.
We can reproduce this issue in our environment.

May I know how do you create the onnx file?
Have you tried to inference it with other frameworks (ex.onnxruntime?)

Thanks.

  1. I have exported the ONNX using “torch.onnx.export”
  2. I have done the inference with PyTorch, OnnxRuntime, and TensorRT. PyTorch and OnnxRuntime give the same result whereas TensorRT is not.

I think some problem with TrtEngine buld from ONNX. I have other Segmentation model as well, that loads fine but the result were not similar. I have tried the polygraphy for segmentation model, Pytorch and OnnRuntime gives same result but the TRT Engine not.

Hi,

We found this might be a regression issue since the model works well on JetPack4.5.
If acceptable, you can use JetPack4.5 as the temporal solution.

We are checking the ‘input mismatched’ issue internally.
Will update more information with you later.

Thanks.

Hi,

This issue is already fixed in TensorRT v7.2.
For JetPack 4.5.1 environment, you can upgrade the nvonnxparser to v7.2 to fix this issue.

Below is the detailed steps for your reference:

1. Install cmake-3.13.5

$ sudo apt-get install -y protobuf-compiler libprotobuf-dev openssl libssl-dev libcurl4-openssl-dev
$ wget https://github.com/Kitware/CMake/releases/download/v3.13.5/cmake-3.13.5.tar.gz
$ tar xvf cmake-3.13.5.tar.gz
$ cd cmake-3.13.5/
$ ./bootstrap --system-curl
$ make -j$(nproc)
$ echo 'export PATH='${PWD}'/bin/:$PATH' >> ~/.bashrc
$ source ~/.bashrc

2. Build onnx-tensorrt

$ git clone https://github.com/onnx/onnx-tensorrt.git
$ cd onnx-tensorrt/
$ git submodule update --init --recursive
$ mkdir build && cd build
$ cmake ../
$ make -j

$ sudo mv /usr/lib/aarch64-linux-gnu/libnvonnxparser.so.7.1.3 libnvonnxparser.so.7.1.3_bk
$ sudo cp libnvonnxparser.so.7.2.2 /usr/lib/aarch64-linux-gnu/libnvonnxparser.so.7.2.2

$ sudo rm /usr/lib/aarch64-linux-gnu/libnvonnxparser.so.7
$ sudo ln -s /usr/lib/aarch64-linux-gnu/libnvonnxparser.so.7.2.2 /usr/lib/aarch64-linux-gnu/libnvonnxparser.so.7
$ sudo ldconfig

Thanks.

JetPack 4.5.1 has TensorRT 7.1
How Can I install TensorRT 7.2 or TensorRT 8.x.x on NX?

Actually, all the problems are resolved on X86_64 Ubuntu using TensorRT 8.0+CUDA 11.3.

I need some help to install TensorRT 8.0 on NX. Could you guide me how to do that.
I have read the DEB installation Installation Guide :: NVIDIA Deep Learning TensorRT Documentation

But there are no repo for ARM64. Could you please help me here?

Hi,

Sorry for the unclear statement.

The root cause is from onnx parser rather than TensorRT library itself.
You will still need to use TensorRT v7.1.3 for compatibility.
But upgrade the ONNX parser to solve this issue.

Thanks.

Found Issue after the solution you have provided on ARM64-NX

TensorRT 7.1.3 installed through JetPack 4.5.1
Onnx-Parser installed as per your instruction 7.2.2

Following the issue

/home/onnx-tensorrt/onnx2trt_utils.cpp:291: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
/home/onnx-tensorrt/ModelImporter.cpp:703: While parsing node number 37 [Resize -> "300"]:
/home/onnx-tensorrt/ModelImporter.cpp:704: --- Begin node ---
/home/onnx-tensorrt/ModelImporter.cpp:705: input: "295"
input: "299"
input: "526"
output: "300"
name: "Resize_37"
op_type: "Resize"
attribute {
  name: "coordinate_transformation_mode"
  s: "align_corners"
  type: STRING
}
attribute {
  name: "cubic_coeff_a"
  f: -0.75
  type: FLOAT
}
attribute {
  name: "mode"
  s: "linear"
  type: STRING
}
attribute {
  name: "nearest_mode"
  s: "floor"
  type: STRING
}

/home/onnx-tensorrt/ModelImporter.cpp:706: --- End node ---
/home/onnx-tensorrt/ModelImporter.cpp:709: ERROR: /home/onnx-tensorrt/builtin_op_importers.cpp:3074 In function importResize:
[8] Assertion failed: (transformationMode == "asymmetric" || transformationMode == "pytorch_half_pixel" || transformationMode == "half_pixel") && "TensorRT only supports half pixel, pytorch half_pixel, and asymmetric tranformation mode for linear resizes when scales are provided!"
kError: Assertion failed: (transformationMode == "asymmetric" || transformationMode == "pytorch_half_pixel" || transformationMode == "half_pixel") && "TensorRT only supports half pixel, pytorch half_pixel, and asymmetric tranformation mode for linear resizes when scales are provided!"
Network must have at least one output
Network validation failed.

Hi,

Do you use the same model that shared with us in this comment.
We didn’t meet other error when verifying the ONNX model.

For your latest error, please check below comment for more informaiton:

Thanks.

This is very helpful! I could able to fix lots of problems with the above solution.

Good to know this!
Does your model work on NX now?

Thanks.

I am facing issue as below, while building onnx parser on TensorRT 7.1.3

/home/aim/Downloads/onnx-tensorrt/builtin_op_importers.cpp:5042:12: error: ‘class nvinfer1::IResizeLayer’ has no member named ‘setSelectorForSinglePixel’
     layer->setSelectorForSinglePixel(selector);
            ^~~~~~~~~~~~~~~~~~~~~~~~~
/home/aim/Downloads/onnx-tensorrt/builtin_op_importers.cpp:5043:12: error: ‘class nvinfer1::IResizeLayer’ has no member named ‘setCoordinateTransformation’
     layer->setCoordinateTransformation(transformation);
            ^~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/aim/Downloads/onnx-tensorrt/builtin_op_importers.cpp:5044:12: error: ‘class nvinfer1::IResizeLayer’ has no member named ‘setNearestRounding’
     layer->setNearestRounding(roundMode);

I am able to resolve this issue. Bu checking out 21.05 branch on onnx