Orin NX 10% slower than Xavier AGX

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin NX
1

Hi there,

I am benchmarking the performance of a 16GB Orin NX versus my previous 16GB Xavier AGX.

I’m running this using trtexec and have noticed the following output. On the Xavier AGX, I achieved a 99th percentile latency of 1.57ms with a compute time of 1.45ms. On the Orin NX, I achieved a 99th percentile latency of 1.78ms with a compute time of 1.66ms.

The commands used to set this up was trtexec —onnx=model.onnx —int8 —duration=10 —avgRuns=100 on both devices. To troubleshoot, I’ve ensured that the both devices have run sudo nvpmodel -m 0 and sudo jetson_clocks. Both GPUs are above 90% utilization, what are the potential reasons for this performance decrease. I would expect the Orin NX to be faster.

Model can be found here: Odin-CV/CV/models/model.onnx at main · SumedhP/Odin-CV · GitHub

3

Hi,

Which TensorRT version do you use for benchmarking?
Could you test it with JetPack 5/TensorRT 8.5?
Since it is available for both OrinNX and AGX Xavier.

Thanks.

4

The Orin NX is already using JetPack 5.1/TensorRT 8.5.
I just updated the Xavier AGX to JetPack5.1.3/TensorRt 8.5.2 (Previous run was using tensorRT 8.4) and following the same testing procedure, have achieved the 99th percentile of latency:1.54ms and GPU Compute: 1.43ms avg over 5 benchmark runs

5

There is no update from you for a period, assuming this is not an issue anymore.
Hence, we are closing this topic. If need further support, please open a new one.
Thanks

Hi,

Could you run it with --verbose and share the output log (both OrinNX and AGX Xavier) with us?

Thanks.