How to Limit AI Performance to Specific TOPS on Jetson Orin Nano Developer Kit

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin Nano
1

Hi there,

I am using the Jetson Orin Nano Developer Kit, which is capable of up to 40 TOPS of AI performance as advertised. For my project related to ML inference, I need to restrict the AI performance to a maximum of 5 TOPS to align with specific requirements.

I was wondering if there is a way to limit or cap the achievable TOPS on the Jetson Orin Nano? If not directly possible, are there recommended methods to limit AI performance? Also, is there a way to determine how many TOPS my code is using?

Thank you!

3

Hi,

Could you share the reason you need to limit the device’s computational performance?
Is it related to the power consumption? If so, we have a tool to limit power consumption:

https://docs.nvidia.com/jetson/archives/r36.4/DeveloperGuide/SD/PlatformPowerAndPerformance/JetsonOrinNanoSeriesJetsonOrinNxSeriesAndJetsonAgxOrinSeries.html#supported-modes-and-power-efficiency

Although we don’t have data for Orin Nano, below is the table of the Orin performance across different power modes for your reference:

Thanks.

4

I am trying to limit the device’s computational performance because we are trying to meet our customer’s requirements to run our code and measure the performance (fps) using a device that has around 5 TOPS. I did try to test our code with the 7W power mode, and the resulting fps is half of what the 15W power mode achieved. Is it safe to assume that the 7W power mode provides up to 20 TOPS in this case?

Also, how did you guys measure the Orin performance across different power modes?

5

Hi,

TOPS from our spec is hardware theoretical number.
To test the TOPS, we recommended to use cutlass library:

Thanks.

6

Okay thanks! I just wanted to ask another quick question, although this is regarding the Jetson Xavier NX Development Kit. Based on the link below, I noticed that the Jetson Xavier NX Development Kit is listed as 6 TFLOPS performance for FP16 operations. While I know it is officially advertised as 21 TOPS for INT8, I’m curious if the 6 TFLOPS figure for FP16 is accurate or realistic. Do you have any insight on this?

https://www.seeedstudio.com/blog/2020/05/19/compare-nvidia-jetson-xavier-nx-with-jetson-tx2-developer-kits/?srsltid=AfmBOooHOnIa7omu1RleLtbNt6V6W7jjM5PthIrX-y3rkZ1HxKnGnXDo

7

Hi,

We don’t have public data for Xavier NX FP16.
But the peak performance of Xavier NX is 21 TOPs.

Thanks.

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.