Issue compiling onnx model to trt using trtexec

Autonomous Vehicles DRIVE AGX Thor DRIVE AGX Thor General
1

DRIVE OS Version: 7.0.3

Issue Description:

We have a D-Fine model which we compile from onnx to trt

On Drive AGX Thor, the only way to get correct results is to compile with –builderOptimizationLevel=0 which results in a very slow model (20+ms). Every other optimization level or swithcing to fp16 results in a trt that does not work.

The exact same onnx if we compile it on the Jetson AGX Orin, works prefectly even with fp16 and default optimization level and results in a fast model (11ms) which works

It seems that on the Thor trtexec does some very aggressive optimizations that break the model

Is this an issue with trtexec? How can we compile the model to get a better time than the Orin and still work?

3

Dear @dan_borlovan ,
Is it possible to share model to see if it is a regression or got fixed in latest TRT ?

4

Sorry we cannot share our model

5

Is it possible to share dummy model?
Please compare the layer times on both using dumpProfile in trrexec to get some insights.
Also,TRT versions on both are not same.

6

Dear @dan_borlovan ,
Could you please provide any update? Is it possible to share dummy model for repro and investigation on our side?

7

I will try to find a free model on which to reproduce the issue and get back

9

There is no update from you for a period, assuming this is not an issue anymore.
Hence, we are closing this topic. If need further support, please open a new one.
Thanks ~0506