Trtexec failed to generate engine (Internal Error) with DLA

Autonomous Machines Jetson & Embedded Systems Jetson Orin NX
, ,
1

Descrption
I tried launching the pre-built trtexec tool to generate trt engine for my onnx model (derived from torchvision Faster RCNN). My computing device is a Jetson Orin NX with two DL accelerators. Engine generation and inference was successful at FP16 precision without DLA. However, when DLA is enabled, trtexec outputs the following error during engine generation:

[09/08/2022-12:15:55] [E] Error[1]: [valueCloner.cpp::replace::31] Error Code 1: Internal Error (Tensor is not known to this ValueCloner)
[09/08/2022-12:15:55] [E] Error[2]: [builder.cpp::buildSerializedNetwork::751] Error Code 2: Internal Error (Assertion engine != nullptr failed. )

The exact command used was:

trtexec --onnx=model.onnx --fp16 --workspace=10000 --useDLACore=0 --allowGPUFallback

Please see attached the complete output of trtexec.
dla_error_log.txt (148.5 KB)

Please help look into this matter. I am not surprised to see many layers not supported by DLA; however, I expected those layers to fall back utilizing the GPU normally.

The same error was shown when I changed the index of useDLACore to 1. There also doesn’t seem to be any similar queries related to this issue.

Many thanks in advance!

Environement
TensorRT Version: 8.5.2.0 (included in JetPack 5.1.2)
GPU Type: Jetson Orin NX (16GB)
CUDA Version: 11.4
CUDNN Version: 8.6
Operating System + Version: Ubuntu 20.04

Relevant Files
Happy to provide my model and more details via private messages.

3

Hi,

If the conversion works with GPU mode but fails in DLA mode with allowGPUFallback flag, then this is not expected.
Please PM the model to me so we can reproduce this issue for checking internally.

Thanks.

4

Thank you very much for your prompt response! I have PM you my onnx model.

5

Hi,

Thanks for your patience.

Confirmed that we can reproduce the same behavior in our environment.
Our internal team is now checking on it.

Will let you know once we get any feedback.

Thanks.

8

any updates on this? I am facing the same issue with tensorrt 8.6.12 (model converts fine with GPU only but got the same error with DLA+Fallback)

1 Like
9

Same on TensorRT 8.6.11, Drive AGX Orin (DRIVE OS 6.0.8.1).
Conversion was success when GPU only.

trtexec --onnx=model.onnx --saveEngine=model_fp16_dla.engine --useDLACore=0 --fp16 --allowGPUFallback --dumpProfile --dumpLayerInfo --dumpOptimizationProfile --verbose

…
[02/22/2024-08:07:59] [E] Error[1]: Tensor is not known to this ValueCloner
[02/22/2024-08:07:59] [E] Error[1]: [valueCloner.cpp::replace::31] Error Code 1: Internal Error (Tensor is not known to this ValueCloner)
[02/22/2024-08:07:59] [E] Engine could not be created from network
[02/22/2024-08:07:59] [E] Building engine failed
[02/22/2024-08:07:59] [E] Failed to create engine from model or file.
[02/22/2024-08:07:59] [E] Engine set up failed

10

For issue on DRIVE AGX Orin, please open your topic at Latest DRIVE AGX Orin/DRIVE AGX Orin General topics - NVIDIA Developer Forums

11

Hi,

Due to the limited resources, this issue will be checked in the next debugging cycle which won’t happen shortly.
Thanks and sorry for any inconvenience.