Cannot build a TensorRT engine for DLA because Constant_output_0 is not supported in DLA

[x2547268390]

Hi guys
I tried to build a TensorRT engine for DLA from the ONNX model with trtexec
But it failed with the following errors
Is there any solution to solve it?
I want to know if this is because I used a custom kernel for torch.nn.functional.conv2d in PyTorch

Machine:
Jetson AGX Orin 64G with JetPack 6.0

command:trtexec --onnx=/home/cast/6_24/small_6_24_1024_rb_round_sim.onnx --saveEngine=/home/cast/6_25/small_6_24_1024_rb_round_sim.trt --shapes=input:32x1x1024x1024 --builderOptimizationLevel=5 --int8 --fp16 --noTF32 --useDLACore=0 --explicitBatch --inputIOFormats=fp16:chw --outputIOFormats=fp16:chw --verbose >dla_verbose_626.txt

Error:
trtexec --onnx=/home/cast/6_24/small_6_24_1024_rb_round_sim.onnx --saveEngine=/home/cast/6_25/small_6_24_1024_rb_round_sim.trt --shapes=input:32x1x1024x1024 --builderOptimizationLevel=5 --int8 --fp16 --noTF32 --useDLACore=0 --explicitBatch --inputIOFormats=fp16:chw --outputIOFormats=fp16:chw --verbose >dla_verbose_626.txt
[06/26/2024-11:15:00] [W] --explicitBatch flag has been deprecated and has no effect!
[06/26/2024-11:15:00] [W] Explicit batch dim is automatically enabled if input model is ONNX or if dynamic shapes are provided when the engine is built.
[06/26/2024-11:15:07] [W] [TRT] onnx2trt_utils.cpp:372: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[06/26/2024-11:15:07] [E] Error[2]: [network.cpp::operator()::2789] Error Code 2: Internal Error (Assertion allowGPUFallback failed. Layer ‘/Constant_output_0’ is not supported on DLA but GPU fallback is not enabled.)
[06/26/2024-11:15:07] [E] Error[4]: [network.cpp::validate::2901] Error Code 4: Internal Error (DLA validation failed)
[06/26/2024-11:15:07] [E] Engine could not be created from network
[06/26/2024-11:15:07] [E] Building engine failed
[06/26/2024-11:15:07] [E] Failed to create engine from model or file.
[06/26/2024-11:15:07] [E] Engine set up failed

[AastaLLL]

Hi,

Please try to add --allowGPUFallback when running the command.

Thanks.

[x2547268390]

Thank you for your response. However, what I need is a TRT file that can work independently on the DLA. My model exports correctly on the GPU, but I hope to obtain a TRT file that can run independently on the DLA. This way, when I run TRT on my GPU, the DLA can also independently work with the TRT files.

[AastaLLL]

Hi,

Could you share the ONNX model or the log with --verbose enabled so we can know more why the layer needs the GPU fallback?

Thanks.

[x2547268390]

Thank you for your response. Attached are the ONNX files and logs for my model
dla_verbose_626.txt (227.7 KB)

small_6_24_1024_rb_round_sim.onnx

[x2547268390]

I wasn’t sure if you received it, so I sent it again.
dla_verbose_626.txt (227.7 KB)
small_6_24_1024_rb_round_sim.onnx - Google 云端硬盘

[AastaLLL]

Hi,

We have received the model.
Will check and provide more info to you later.

Thanks.

[AastaLLL]

Hi,

Sorry for the late update.

Have you verified your model with other frameworks like ONNXRuntime?
When we check your model, trtexec reports errors and the model definition does not correct itself.

...
[07/23/2024-03:49:34] [W] Dynamic dimensions required for input: input, but no shapes were provided. Automatically overriding shape to: 1x1x1024x1024
[07/23/2024-03:49:34] [E] Error[4]: [graphShapeAnalyzer.cpp::analyzeShapes::2021] Error Code 4: Miscellaneous (IShuffleLayer /Reshape: reshape changes volume. Reshaping [1,576,64,64] to [3,32,192,64,64].)
[07/23/2024-03:49:34] [E] Engine could not be created from network
...

Thanks.