Jetson Xavier GPU performance drop after inactivity

Robotics & Edge Computing Jetson & Embedded Systems Jetson Xavier NX
1

Hello NVidia team,

We are experiencing a performance drop of NN model inference after inactivity.

After the model is loaded and we do inferences in a loop using TensorRT (via ONNX RT), we are able to achieve a performance of 20ms. But after for example 1 second of inactivity the performance drops to 163 ms. Next immediate inference is faster and eventually we got back to the 20ms.

Something similar is happening with CUDA execution provider.

We are currently using MODE_15W_6CORE mode on Jetson NX Xavier:

mlipovsky@srv-ipjetson-3n:~$ /usr/sbin/nvpmodel -q
NV Fan Mode:quiet
NV Power Mode: MODE_15W_6CORE

Which mode should we use or what are the options to eliminate the performance drop after GPU inactivity?

Thanks

3

Hi,

NV Power mode is used for changing the maximal/minimum hardware frequency.

Please note that by default GPU is in the dynamic mode.
It will automatically switch the frequency between max and min for saving power.

If you want to lock the performance to the maximal.
Please run the following script after setting the nvpmodel.

$ sudo jetson_clocks

Thanks.

4

Hi,

Thank you, it seems it helped, we will test further.
I will get back to you in case the problem persists.

Just to be sure, this setting has no impact on the Jetson device lifetime, right?

Thanks again.
Regards

5

Hi,

YES. It just changes the clock mode from dynamic into fixed.

Thanks.

6

Great, thank you for help.

8

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.