Failed to convert YOLOv7.onnx model to DLA engine

dawn1206 · May 6, 2023, 2:37am

Hello!

I’m trying to convert a yolov7-tiny onnx model to a DLA engine but the process failed.

Error part in log :

Module_id 33 Severity 2 : NVMEDIA_DLA 2493
Module_id 33 Severity 2 : Runtime: loadBare failed. error:  0x000004
[05/06/2023-10:16:52] [V] [TRT] Deleting timing cache: 1348 entries, 1472 hits
[05/06/2023-10:16:52] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 1402, GPU 7362 (MiB)
[05/06/2023-10:16:52] [E] Error[1]: [nvdlaUtils.cpp::deserialize::164] Error Code 1: DLA (NvMediaDlaLoadLoadable : load loadable failed.)
[05/06/2023-10:16:52] [E] Error[2]: [builder.cpp::buildSerializedNetwork::417] Error Code 2: Internal Error (Assertion enginePtr != nullptr failed.)
Segmentation fault (core dumped)

log file:

error.log (1.5 MB)

onnx model file:

yolov7-tiny.onnx (23.8 MB)

System info:

NVIDIA Jetson AGX Xavier [16GB]

Jetpack 4.6 [L4T 32.6.1]

Libraries:

CUDA: 10.2.300
cuDNN: 8.2.1.32
TensorRT: 8.0.1.6

Steps to reproduce

Original pytorch .pt model file is from GitHub - WongKinYiu/yolov7: Implementation of paper - YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors.
Onnx model file is converted from the .pt file using the command:

python export.py --weights ./yolov7-tiny.pt --grid --end2end --simplify --topk-all 100 --iou-thres 0.65 --conf-thres 0.35 --img-size 640 640

Try to convert the onnx model to trt engine executing on DLA using the command:

/usr/src/tensorrt/bin/trtexec --onnx=yolov7-tiny.onnx --saveEngine=yolov7-tiny-nms-dla.trt --fp16 --useDLACore=0 --allowGPUFallback --verbose

AastaLLL · May 8, 2023, 3:06am

Hi,

Please upgrade to the latest JetPack 5.1.1 and try it again.
We can your model with the new TensorRT 8.5:

$ /usr/src/tensorrt/bin/trtexec --onnx=./yolov7-tiny.onnx --useDLACore=0 --allowGPUFallback
&&&& RUNNING TensorRT.trtexec [TensorRT v8502] # /usr/src/tensorrt/bin/trtexec --onnx=./yolov7-tiny.onnx --useDLACore=0 --allowGPUFallback
...
[05/08/2023-11:00:34] [I] Start parsing network model
[05/08/2023-11:00:34] [I] [TRT] ----------------------------------------------------------------
[05/08/2023-11:00:34] [I] [TRT] Input filename:   ./yolov7-tiny.onnx
[05/08/2023-11:00:34] [I] [TRT] ONNX IR version:  0.0.7
[05/08/2023-11:00:34] [I] [TRT] Opset version:    12
[05/08/2023-11:00:34] [I] [TRT] Producer name:    pytorch
[05/08/2023-11:00:34] [I] [TRT] Producer version: 2.0.0
[05/08/2023-11:00:34] [I] [TRT] Domain:           
[05/08/2023-11:00:34] [I] [TRT] Model version:    0
[05/08/2023-11:00:34] [I] [TRT] Doc string:       
[05/08/2023-11:00:34] [I] [TRT] ----------------------------------------------------------------
[05/08/2023-11:00:34] [W] [TRT] onnx2trt_utils.cpp:375: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[05/08/2023-11:00:34] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[05/08/2023-11:00:34] [I] [TRT] No importer registered for op: EfficientNMS_TRT. Attempting to import as plugin.
[05/08/2023-11:00:34] [I] [TRT] Searching for plugin: EfficientNMS_TRT, plugin_version: 1, plugin_namespace: 
[05/08/2023-11:00:34] [I] [TRT] Successfully created plugin: EfficientNMS_TRT
[05/08/2023-11:00:34] [I] Finish parsing network model
[05/08/2023-11:00:34] [W] [TRT] /model/model.32/m/MaxPool: DLA only supports windows in the range of [1-8].
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.32/m/MaxPool' (POOLING): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] /model/model.33/m/MaxPool: DLA only supports windows in the range of [1-8].
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.33/m/MaxPool' (POOLING): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Reshape' (SHUFFLE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Transpose' (SHUFFLE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] /model/model.77/Split: DLA only supports slicing 4 dimensional tensors.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Split' (SLICE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] /model/model.77/Split_9: DLA only supports slicing 4 dimensional tensors.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Split_9' (SLICE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] /model/model.77/Split_10: DLA only supports slicing 4 dimensional tensors.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Split_10' (SLICE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Constant_1_output_0' (CONSTANT): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '(Unnamed Layer* 140) [Shuffle]' (SHUFFLE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Constant_2_output_0' (CONSTANT): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Constant_3_output_0' (CONSTANT): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '(Unnamed Layer* 145) [Shuffle]' (SHUFFLE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] /model/model.77/Pow: DLA cores do not support POW ElementWise operation.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Pow' (ELEMENTWISE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Constant_4_output_0' (CONSTANT): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] /model/model.77/Concat: DLA only supports concatenation on the C dimension.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Concat' (CONCATENATION): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Reshape_1' (SHUFFLE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Reshape_2' (SHUFFLE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Transpose_1' (SHUFFLE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] /model/model.77/Split_1: DLA only supports slicing 4 dimensional tensors.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Split_1' (SLICE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] /model/model.77/Split_1_11: DLA only supports slicing 4 dimensional tensors.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Split_1_11' (SLICE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] /model/model.77/Split_1_12: DLA only supports slicing 4 dimensional tensors.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Split_1_12' (SLICE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Constant_7_output_0' (CONSTANT): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '(Unnamed Layer* 159) [Shuffle]' (SHUFFLE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Constant_8_output_0' (CONSTANT): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '(Unnamed Layer* 163) [Shuffle]' (SHUFFLE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] /model/model.77/Pow_1: DLA cores do not support POW ElementWise operation.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Pow_1' (ELEMENTWISE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Constant_10_output_0' (CONSTANT): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] /model/model.77/Concat_1: DLA only supports concatenation on the C dimension.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Concat_1' (CONCATENATION): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Reshape_3' (SHUFFLE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Reshape_4' (SHUFFLE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Transpose_2' (SHUFFLE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] /model/model.77/Split_2: DLA only supports slicing 4 dimensional tensors.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Split_2' (SLICE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] /model/model.77/Split_2_13: DLA only supports slicing 4 dimensional tensors.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Split_2_13' (SLICE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] /model/model.77/Split_2_14: DLA only supports slicing 4 dimensional tensors.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Split_2_14' (SLICE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Constant_13_output_0' (CONSTANT): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '(Unnamed Layer* 177) [Shuffle]' (SHUFFLE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Constant_14_output_0' (CONSTANT): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '(Unnamed Layer* 181) [Shuffle]' (SHUFFLE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] /model/model.77/Pow_2: DLA cores do not support POW ElementWise operation.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Pow_2' (ELEMENTWISE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Constant_16_output_0' (CONSTANT): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] /model/model.77/Concat_2: DLA only supports concatenation on the C dimension.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Concat_2' (CONCATENATION): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Reshape_5' (SHUFFLE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] /model/model.77/Concat_3: DLA only supports concatenation on the C dimension.
[05/08/2023-11:00:34] [W] [TRT] Layer '/model/model.77/Concat_3' (CONCATENATION): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] /end2end/Slice: DLA only supports slicing 4 dimensional tensors.
[05/08/2023-11:00:34] [W] [TRT] Layer '/end2end/Slice' (SLICE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] /end2end/Slice_1: DLA only supports slicing 4 dimensional tensors.
[05/08/2023-11:00:34] [W] [TRT] Layer '/end2end/Slice_1' (SLICE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] /end2end/Slice_2: DLA only supports slicing 4 dimensional tensors.
[05/08/2023-11:00:34] [W] [TRT] Layer '/end2end/Slice_2' (SLICE): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:34] [W] [TRT] Layer '/end2end/EfficientNMS_TRT' (PLUGIN_V2): Unsupported on DLA. Switching this layer's device type to GPU.
[05/08/2023-11:00:35] [W] [TRT] DLA only allows inputs of the same dimensions to Elementwise, but input shapes were: [1,3,80,80,2] and [1,1,80,80,2]
[05/08/2023-11:00:35] [W] [TRT] Validation failed for DLA layer: /model/model.77/Add. Switching to GPU fallback.
[05/08/2023-11:00:35] [W] [TRT] Splitting DLA subgraph at: /model/model.77/Add because DLA validation failed for this layer.
[05/08/2023-11:00:35] [W] [TRT] DLA only allows inputs of the same dimensions to Elementwise, but input shapes were: [1,3,80,80,2] and [1,1,80,80,2]
[05/08/2023-11:00:35] [W] [TRT] Validation failed for DLA layer: /model/model.77/Add. Switching to GPU fallback.
[05/08/2023-11:00:35] [W] [TRT] DLA only allows inputs of the same dimensions to Elementwise, but input shapes were: [1,3,80,80,2] and [1,3,1,1,2]
[05/08/2023-11:00:35] [W] [TRT] Validation failed for DLA layer: /model/model.77/Mul_1. Switching to GPU fallback.
[05/08/2023-11:00:35] [W] [TRT] DLA only allows inputs of the same dimensions to Elementwise, but input shapes were: [1,3,40,40,2] and [1,1,40,40,2]
[05/08/2023-11:00:35] [W] [TRT] Validation failed for DLA layer: /model/model.77/Add_1. Switching to GPU fallback.
[05/08/2023-11:00:35] [W] [TRT] Splitting DLA subgraph at: /model/model.77/Add_1 because DLA validation failed for this layer.
[05/08/2023-11:00:35] [W] [TRT] DLA only allows inputs of the same dimensions to Elementwise, but input shapes were: [1,3,40,40,2] and [1,1,40,40,2]
[05/08/2023-11:00:35] [W] [TRT] Validation failed for DLA layer: /model/model.77/Add_1. Switching to GPU fallback.
[05/08/2023-11:00:35] [W] [TRT] DLA only allows inputs of the same dimensions to Elementwise, but input shapes were: [1,3,40,40,2] and [1,3,1,1,2]
[05/08/2023-11:00:35] [W] [TRT] Validation failed for DLA layer: /model/model.77/Mul_3. Switching to GPU fallback.
[05/08/2023-11:00:35] [W] [TRT] DLA only allows inputs of the same dimensions to Elementwise, but input shapes were: [1,3,20,20,2] and [1,1,20,20,2]
[05/08/2023-11:00:35] [W] [TRT] Validation failed for DLA layer: /model/model.77/Add_2. Switching to GPU fallback.
[05/08/2023-11:00:35] [W] [TRT] Splitting DLA subgraph at: /model/model.77/Add_2 because DLA validation failed for this layer.
[05/08/2023-11:00:35] [W] [TRT] DLA only allows inputs of the same dimensions to Elementwise, but input shapes were: [1,3,20,20,2] and [1,1,20,20,2]
[05/08/2023-11:00:35] [W] [TRT] Validation failed for DLA layer: /model/model.77/Add_2. Switching to GPU fallback.
[05/08/2023-11:00:35] [W] [TRT] DLA only allows inputs of the same dimensions to Elementwise, but input shapes were: [1,3,20,20,2] and [1,3,1,1,2]
[05/08/2023-11:00:35] [W] [TRT] Validation failed for DLA layer: /model/model.77/Mul_5. Switching to GPU fallback.
[05/08/2023-11:00:35] [W] [TRT] Input tensor has less than 4 dimensions for /end2end/Mul. At least one shuffle layer will be inserted which cannot run on DLA.
[05/08/2023-11:00:35] [W] [TRT] Dimension: 2 (25200) exceeds maximum allowed size for DLA: 8192
[05/08/2023-11:00:35] [W] [TRT] Validation failed for DLA layer: /end2end/Mul. Switching to GPU fallback.
[05/08/2023-11:00:36] [I] [TRT] ---------- Layers Running on DLA ----------
[05/08/2023-11:00:36] [I] [TRT] [DlaLayer] {ForeignNode[/model/model.0/conv/Conv.../model/model.30/act/LeakyRelu]}
[05/08/2023-11:00:36] [I] [TRT] [DlaLayer] {ForeignNode[/model/model.29/conv/Conv.../model/model.77/m.2/Conv]}
[05/08/2023-11:00:36] [I] [TRT] [DlaLayer] {ForeignNode[/model/model.77/Sigmoid.../model/model.77/Sigmoid_2]}
[05/08/2023-11:00:36] [I] [TRT] [DlaLayer] {ForeignNode[/model/model.77/Constant_1_output_0 + (Unnamed Layer* 140) [Shuffle] + /model/model.77/Mul]}
[05/08/2023-11:00:36] [I] [TRT] [DlaLayer] {ForeignNode[/model/model.77/Constant_7_output_0 + (Unnamed Layer* 159) [Shuffle] + /model/model.77/Mul_2]}
[05/08/2023-11:00:36] [I] [TRT] [DlaLayer] {ForeignNode[/model/model.77/Constant_13_output_0 + (Unnamed Layer* 177) [Shuffle] + /model/model.77/Mul_4]}
[05/08/2023-11:00:36] [I] [TRT] ---------- Layers Running on GPU ----------
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] POOLING: /model/model.32/m/MaxPool
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] POOLING: /model/model.33/m/MaxPool
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] SHUFFLE: /model/model.77/Reshape + /model/model.77/Transpose
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] SHUFFLE: /model/model.77/Reshape_2 + /model/model.77/Transpose_1
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] SHUFFLE: /model/model.77/Reshape_4 + /model/model.77/Transpose_2
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] SLICE: /model/model.77/Split
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] SLICE: /model/model.77/Split_9
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] CONSTANT: /model/model.77/Constant_2_output_0
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] CONSTANT: /model/model.77/Constant_4_output_0
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] SLICE: /model/model.77/Split_1
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] SLICE: /model/model.77/Split_1_11
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] CONSTANT: /model/model.77/Constant_8_output_0
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] CONSTANT: /model/model.77/Constant_10_output_0
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] SLICE: /model/model.77/Split_2
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] SLICE: /model/model.77/Split_2_13
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] CONSTANT: /model/model.77/Constant_14_output_0
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] CONSTANT: /model/model.77/Constant_16_output_0
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] ELEMENTWISE: /model/model.77/Add
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] POINTWISE: PWN(PWN(/model/model.77/Constant_3_output_0 + (Unnamed Layer* 145) [Shuffle], /model/model.77/Pow), /model/model.77/Mul_1)
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] ELEMENTWISE: /model/model.77/Add_1
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] POINTWISE: PWN(PWN(/model/model.77/Constant_3_output_0_clone_1 + (Unnamed Layer* 163) [Shuffle], /model/model.77/Pow_1), /model/model.77/Mul_3)
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] ELEMENTWISE: /model/model.77/Add_2
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] POINTWISE: PWN(PWN(/model/model.77/Constant_3_output_0_clone_2 + (Unnamed Layer* 181) [Shuffle], /model/model.77/Pow_2), /model/model.77/Mul_5)
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] SLICE: /model/model.77/Split_10
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] SHUFFLE: /model/model.77/Reshape_1
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] COPY: /model/model.77/Reshape_1_copy_output
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] SLICE: /model/model.77/Split_1_12
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] SHUFFLE: /model/model.77/Reshape_3
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] COPY: /model/model.77/Reshape_3_copy_output
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] SLICE: /model/model.77/Split_2_14
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] SHUFFLE: /model/model.77/Reshape_5
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] COPY: /model/model.77/Reshape_5_copy_output
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] ELEMENTWISE: /end2end/Mul
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] COPY: /end2end/Slice
[05/08/2023-11:00:36] [I] [TRT] [GpuLayer] PLUGIN_V2: /end2end/EfficientNMS_TRT
[05/08/2023-11:00:41] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +535, GPU +788, now: CPU 1134, GPU 6091 (MiB)
[05/08/2023-11:00:42] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +82, GPU +139, now: CPU 1216, GPU 6230 (MiB)
[05/08/2023-11:00:42] [I] [TRT] Local timing cache in use. Profiling results in this builder pass will not be stored.
[05/08/2023-11:02:44] [I] [TRT] Total Activation Memory: 32120889856
[05/08/2023-11:02:44] [I] [TRT] Detected 1 inputs and 4 output network tensors.
[05/08/2023-11:02:44] [I] [TRT] Total Host Persistent Memory: 4720
[05/08/2023-11:02:44] [I] [TRT] Total Device Persistent Memory: 0
[05/08/2023-11:02:44] [I] [TRT] Total Scratch Memory: 40320768
[05/08/2023-11:02:44] [I] [TRT] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 12 MiB, GPU 769 MiB
[05/08/2023-11:02:44] [I] [TRT] [BlockAssignment] Started assigning block shifts. This will take 57 steps to complete.
[05/08/2023-11:02:44] [I] [TRT] [BlockAssignment] Algorithm ShiftNTopDown took 4.00517ms to assign 10 blocks to 57 nodes requiring 49952256 bytes.
[05/08/2023-11:02:44] [I] [TRT] Total Activation Memory: 49952256
[05/08/2023-11:02:44] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 1248, GPU 9902 (MiB)
[05/08/2023-11:02:44] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +0, GPU +0, now: CPU 1248, GPU 9902 (MiB)
[05/08/2023-11:02:44] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in building engine: CPU +12, GPU +4, now: CPU 12, GPU 4 (MiB)
[05/08/2023-11:02:44] [I] Engine built in 134.229 sec.
[05/08/2023-11:02:45] [I] [TRT] Loaded engine size: 12 MiB
[05/08/2023-11:02:45] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 941, GPU 9756 (MiB)
[05/08/2023-11:02:45] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +0, GPU +0, now: CPU 941, GPU 9756 (MiB)
[05/08/2023-11:02:45] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +12, GPU +0, now: CPU 12, GPU 0 (MiB)
[05/08/2023-11:02:45] [I] Engine deserialized in 0.0163689 sec.
[05/08/2023-11:02:45] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 941, GPU 9756 (MiB)
[05/08/2023-11:02:45] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +0, GPU +0, now: CPU 941, GPU 9756 (MiB)
[05/08/2023-11:02:45] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +47, now: CPU 12, GPU 47 (MiB)
[05/08/2023-11:02:45] [I] Setting persistentCacheLimit to 0 bytes.
[05/08/2023-11:02:45] [I] Using random values for input images
[05/08/2023-11:02:45] [I] Created input binding for images with dimensions 1x3x640x640
[05/08/2023-11:02:45] [I] Using random values for output num_dets
[05/08/2023-11:02:45] [I] Created output binding for num_dets with dimensions 1x1
[05/08/2023-11:02:45] [I] Using random values for output det_boxes
[05/08/2023-11:02:45] [I] Created output binding for det_boxes with dimensions 1x80x4
[05/08/2023-11:02:45] [I] Using random values for output det_scores
[05/08/2023-11:02:45] [I] Created output binding for det_scores with dimensions 1x80
[05/08/2023-11:02:45] [I] Using random values for output det_classes
[05/08/2023-11:02:45] [I] Created output binding for det_classes with dimensions 1x80
[05/08/2023-11:02:45] [I] Starting inference
...
&&&& PASSED TensorRT.trtexec [TensorRT v8502] # /usr/src/tensorrt/bin/trtexec --onnx=./yolov7-tiny.onnx --useDLACore=0 --allowGPUFallback

Thanks.

dawn1206 · May 8, 2023, 4:59am

Thank you for your reply, but I still want to know the reasons why the problem happened in my environment. Can you tell me the possible causes of this issue so that I can complete this operation in this environment?

dawn1206 · May 8, 2023, 5:18am

Because the JetPack environment is related to other drivers, it’s unlikely that I can change the JetPack version for this task. I really appreciate that you can help me understand and solve these issues.

AastaLLL · May 9, 2023, 5:06am

Hi,

The error looks similar to the below topic:

The error is caused by an internal DLA bug in the concat layer.
Thanks.

dawn1206 · May 9, 2023, 5:23am

Thanks for your reply!

Topic		Replies	Views
Error when use DLA with yoloV4 tensorRT Jetson AGX Xavier tensorrt , dla	6	1107	April 26, 2023
Trtexec log problem and use DLA error on Jetson Xavier Jetson AGX Xavier dla	6	1729	March 23, 2021
Cannot build a TensorRT engine for DLA from a large ONNX file Jetson Xavier NX tensorrt , nvbugs , dla	11	2869	July 21, 2021
Failed to generate DLA Engine from Inception-v3 model Jetson AGX Xavier tensorrt , nvbugs , dla	5	1343	December 23, 2021
[TensorRT] Running a simple onnx model on Jetson Xavier DLA Jetson Xavier NX tensorrt , onnx	11	3444	August 10, 2022
Unable to run model only in DLA (without GPU) TensorRT dla	5	607	March 28, 2024
Deepstream-Yolo doesn't use DLA DeepStream SDK	19	1079	June 25, 2024
Engine creation fails when using DLA with GPU fallback Jetson AGX Xavier tensorrt , dla	10	2316	March 22, 2022
Running yolox with relu activation on dla gives me incorrect result Jetson Xavier NX dla	5	1078	May 30, 2022
Using dla on orin nx meet an error Jetson AGX Xavier dla	8	412	September 8, 2024