Unable to generate yolov3 or DINO model engine file from onnx file

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

Unable to generate engine files from onnx files for models like yolov3 and DINO

Environment

TensorRT Version: 10.0.1.6
GPU Type: Tesla T4
Nvidia Driver Version: 535.171.04
CUDA Version: 12.2
Operating System + Version: Ubuntu 20.04.6 LTS
Python Version (if applicable): 3.8.10

Relevant Files

I used the TENSORRT quick start guide example to run trtexec tool to try out generating engine files for other models than shown in the example.

This section particularly.

image
image1811×413 57.6 KB

I was able to run the example and generate engine file as shown here. But when I tried with other models, I received errors.

Steps To Reproduce

root@67ccd431bbd1:/workspace# trtexec --onnx=yolov3u.onnx --saveEngine=yolov3u.trt

found these errors,

[05/30/2024-08:27:37] [E] [TRT] /model.28/dfl/Reshape: volume mismatch. Input dimensions [1,72,8400] have volume 604800 and output dimensions [1,4,16,8400] have volume 537600.
[05/30/2024-08:27:37] [E] [TRT] /model.28/dfl/Reshape: volume mismatch. Input dimensions [1,72,8400] have volume 604800 and output dimensions [1,4,16,8400] have volume 537600.
[05/30/2024-08:27:37] [E] [TRT] /model.28/dfl/Reshape: volume mismatch. Input dimensions [1,72,8400] have volume 604800 and output dimensions [1,4,16,8400] have volume 537600.
[05/30/2024-08:27:37] [E] [TRT] /model.28/dfl/Reshape: volume mismatch. Input dimensions [1,72,8400] have volume 604800 and output dimensions [1,4,16,8400] have volume 537600.
[05/30/2024-08:27:37] [E] [TRT] /model.28/dfl/Reshape: volume mismatch. Input dimensions [1,72,8400] have volume 604800 and output dimensions [1,4,16,8400] have volume 537600.
[05/30/2024-08:27:37] [E] [TRT] /model.28/dfl/Reshape: volume mismatch. Input dimensions [1,72,8400] have volume 604800 and output dimensions [1,4,16,8400] have volume 537600.
[05/30/2024-08:27:37] [E] [TRT] /model.28/dfl/Reshape: volume mismatch. Input dimensions [1,72,8400] have volume 604800 and output dimensions [1,4,16,8400] have volume 537600.
While parsing node number 302 [Softmax]:
ERROR: /workspace/TensorRT/parsers/onnx/onnx2trt_utils.cpp:188 In function convertAxis:
[8] Assertion failed: axis >= 0 && axis < nbDims
[05/30/2024-08:27:37] [E] Failed to parse onnx file
[05/30/2024-08:27:37] [E] Parsing model failed
[05/30/2024-08:27:37] [E] Engine creation failed
[05/30/2024-08:27:37] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec # trtexec --onnx=yolov3u.onnx --saveEngine=yolov3u.trt

Am I missing anything here in the high-level?

2

It seems that your model has some issues with the definition.
Could you run the ONNX model with other frameworks, ex onnxruntime?

Thanks.