Performance issues when upgrading to JetPack 5

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
,
9

Hi,

Thanks for the source and package. Here are some findings.
We have tried the engine. As we don’t have torch_tensorrt before, we run the model with the TensorRT binary directly.

$/usr/src/tensorrt/bin/trtexec --loadEngine=./mobilenet_v2.[ver].engine --iterations=100

With the TensorRT binary, we don’t observe the perf drop issue:

JetPack 4: 5.31199 ms
JetPack 5: 5.26338 ms

Since TensorRT runs the model with fake input (fast device memory), the regression might be memory-related.
(For example, a known memory regression caused by the security update:
Bad memory performance on JetPack 5.0.2 (5.10.104-tegra))

We will try the wheel and check if we can reproduce the issue locally to gather more info.
Could you also test the same command for the real model you used to make sure there is no degradation on the TensorRT side?

Thanks.