ONNX to TensoRT conversion failing with error: "each train expected to have at most one ShapeHostToDeviceNode"

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
,
1

Description

ONNX file was generated from PyTorch Retinanet and then folded using polygraphy. Then when running /usr/src/tensorrt/bin/trtexec --onnx=folded.onnx --saveEngine=model.engine, this is the error:

...
[07/11/2023-17:14:31] [I] [TRT] [GpuLayer] MYELIN: {ForeignNode[onnx::Equal_3245.../model/Concat_92]}
[07/11/2023-17:14:31] [I] [TRT] [GpuLayer] TRAIN_STATION: [trainStation3]
[07/11/2023-17:14:31] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 913, GPU 5828 (MiB)
[07/11/2023-17:14:31] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +1, GPU +0, now: CPU 914, GPU 5828 (MiB)
[07/11/2023-17:14:31] [I] [TRT] Local timing cache in use. Profiling results in this builder pass will not be stored.
[07/11/2023-17:14:31] [E] Error[2]: [injectImplicitPadding.cpp::grabShapeHostToDeviceNodes::419] Error Code 2: Internal Error (Assertion !holder failed. each train expected to have at most one ShapeHostToDeviceNode)
[07/11/2023-17:14:31] [E] Error[2]: [builder.cpp::buildSerializedNetwork::751] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
[07/11/2023-17:14:31] [E] Engine could not be created from network
[07/11/2023-17:14:31] [E] Building engine failed
[07/11/2023-17:14:31] [E] Failed to create engine from model or file.
[07/11/2023-17:14:31] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8502] # /usr/src/tensorrt/bin/trtexec --onnx=folded.onnx --saveEngine=model.engine

Any ideas on what is causing this?

Environment

TensorRT Version: 8.5
GPU Type: Jetson Xavier
Nvidia Driver Version:
CUDA Version: 11.4
CUDNN Version:
Operating System + Version: Ubuntu 20
Python Version (if applicable): 3.8.10
TensorFlow Version (if applicable):
PyTorch Version (if applicable): 1.13
Baremetal or Container (if container which image + tag):

Relevant Files

Steps To Reproduce

Using that onnx, run /usr/src/tensorrt/bin/trtexec --onnx=folded.onnx --saveEngine=model.engine

2

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.

In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

4

Hi,

We recommend that you please try the latest TensorRT version 8.6.1. Please share with us your complete verbose logs if you still face the issue.

You can also try TensorRT | NVIDIA NGC container for easy setup.

Thank you.

5

Is it possible to use TensorRT 8.6.1 on Jetson Xavier with Jetpack 5.1? I thought I saw some indications that this is not possible

6

Hi,

We are moving this post to the Jetson Xavier forum to get better help on the above query.

Thank you.

8

Hi,

TensorRT 8.6 is not available for Jetson yet.
Let’s check if the error comes from ONNX or TensorRT first.

Have you run the ONNX file with other frameworks like ONNXRuntime?
If not, please do so.

Thanks.

9

I removed NMS from the Pytorch model and that seems to have resolved this error. The engine is now fully building. However, I am getting a seg fault when trying to run the example inference code found here: https://github.com/NVIDIA/TensorRT/blob/main/quickstart/SemanticSegmentation/tutorial-runtime.ipynb

Traceback (most recent call last):
  File "run.py", line 85, in <module>
    infer(engine, input_file, output_file)
  File "run.py", line 67, in infer
    output_memory = cuda.mem_alloc(output_buffer.nbytes)
pycuda._driver.LogicError: cuMemAlloc failed: invalid argument
terminate called after throwing an instance of 'nvinfer1::plugin::CudnnError'
  what():  std::exception
Aborted (core dumped)

The error seems to stem from the size of one of the bindings being 0. Any idea why? Is there a more up to date inference example I should be using?

10

Hi,

Could you try if the engine file can work with trtexec tool first?
Below is another inference example for your reference:
https://elinux.org/Jetson/L4T/TRT_Customized_Example#OpenCV_with_PLAN_model

Thanks.

11

It does work with trtexec tool. I was able to get if working with that inference code by modifying the model so its output was not dynamically shaped

13

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.