How to check performance of TensorRT optimised file

Autonomous Vehicles DRIVE AGX Xavier DRIVE AGX Xavier General
1

Please provide the following info (check/uncheck the boxes after creating this topic):
Software Version
DRIVE OS Linux 5.2.6
DRIVE OS Linux 5.2.6 and DriveWorks 4.0
DRIVE OS Linux 5.2.0
DRIVE OS Linux 5.2.0 and DriveWorks 3.5
NVIDIA DRIVE™ Software 10.0 (Linux)
NVIDIA DRIVE™ Software 9.0 (Linux)
other DRIVE OS version
other

Target Operating System
Linux
QNX
other

Hardware Platform
NVIDIA DRIVE™ AGX Xavier DevKit (E3550)
NVIDIA DRIVE™ AGX Pegasus DevKit (E3550)
other

SDK Manager Version
1.9.1.10844
other

Host Machine Version
native Ubuntu 18.04
other

I had onnx model and optimised the model using TensorRT optimisation tool.
Now I am planning to compare onnx and TensorRT optimised file.
How to compare?

3

Dear @alksainath.medam,
You can use trtexec to know total execution(inference) time and per layer perf timings of TRT model.

4

ok @SivaRamaKrishnaNV Thanks
Any tutorials how to use trtexec to know this execution time and per layer perf timing of TRT model.

I had used below link to optimise model.
https://developer.nvidia.com/docs/drive/driveworks/latest/nvsdk_dw_html/dwx_tensorRT_tool.html

5

Dear @alksainath.medam,
Could you check /usr/src/tensorrt/bin/trtexec --help to know the profiling options?

1 Like
6

Thanks @SivaRamaKrishnaNV
I am attaching the log file of profiling options and sharing the onnx as well as trt optimised model.
I just want to compare these two models in terms of accuracy or inference time, in any comparison means as possible.
model2cnn.trt (4.0 MB)
model2cnn.onnx (1.9 MB)

7

Dear @alksainath.medam,
I don’t see trtexec profiling options log attached.

8

Sorry @SivaRamaKrishnaNV
trtexe_help (2.3 KB)
Attached now.

9

Dear @alksainath.medam,
We have included --dumpProfile option with trtexec to report per layer perf information which seems to be missing. Could you check if --dumpProfile option works on DRIVE SW 10.0 release.
Regarding accuracy and inference time comparison, we don’t have any tools/scripts for comparing ONNX vs TRT. You may have to write your scripts on own to compare the performance on a dataset.

10

Okay @SivaRamaKrishnaNV
Checked --dumpProfile option getting error as unknown argument.
Attaching the log file.
for TRT performance any code is there to check performance?

11

Dear @alksainath.medam,
Is it possible to upgrade to DRIVE OS 5.2.6 + DW 4.0 release. It has --dumpProfile option.

12

Thanks @SivaRamaKrishnaNV will check with my team.

13

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.