System throttled due to over-current?

hi all,

we’re still investigate this issue internally, thanks

Can you give any updates how Nvidia is correcting performance issues ?

I know internal issues cant be said upfront but i guess we customers are eager to learn any positive corrective changes info we might get

Late to the party, however I’m getting the same throttling issue with the Xavier NX dev. kit running Jetpack 4.5 and the standard 19V 2.73A power brick. I saw this first while using the Stereolabs ZED SDK tools (ZED_Depth_Viewer & ZEDfu) while testing with a ZED Mini. I also didn’t see this issue with previous Jetpack versions.
Thanks for looking into this Nvidia, hopefully the internal investigations are going well.

Happened to me during visual recognition from camera, and also during ROS installation in 15w 6core mode. Previously did not happen.

hi all,

could you please have the workaround to increase instantaneous OC limit, from 3.6A to 5A;
this 5A workaround should still acceptable with hardware aspect.
please have below command to set 5A instantaneous OC limit:
for example,
# echo 5000 > /sys/devices/c250000.i2c/i2c-7/7-0040/iio:device0/crit_current_limit_0

looking forward your feedbacks whether this approach works or not.
thanks!

2 Likes

Device Info: Jetson Xavier NX (JetPack 4.5.0)
I changed the permissions, wrote 5000 and restore permissions

$ cd /sys/devices/c250000.i2c/i2c-7/7-0040/iio\:device0/
$ cat crit_current_limit_0
3600 ma
$ sudo chmod 666 crit_current_limit_0 
$ ls -l crit_current_limit_0
-rw-rw-rw- 1 root root 4096 Mar 25 16:10 crit_current_limit_0
$ sudo echo -n "5000" > crit_current_limit_0
$ sudo cat crit_current_limit_0
5000 ma
$ sudo chmod 600 crit_current_limit_0 
$ ls -l crit_current_limit_0
-rw------- 1 root root 4096 Mar 25 16:16 crit_current_limit_0
$ sudo cat crit_current_limit_0
5000 ma

I tried TensorFlow in Docker container. It looks worked!!!

Thanks @JerryChang

4 Likes

Hi JerryChang, mehmetdeniz, worked for me too - no more throttling.
Thanks!

Hi again JerryChang, could you give a tip on how to make this workaround persistent - to work after a reboot?

Thanks

Unfortunately for me i still keep getting warning (all 15W 2/4/6) modes and after acknowledging it keeps coming back

/sys/devices/c250000.i2c/i2c-7/7-0040/iio:device0$ sudo more crit_current_limit_0
5000 ma

Thanks that has worked for me.

hello Kermit,

you may have a workaround by updating /etc/systemd/nv.sh, to include the command into initial scripts.

3 Likes

Hi Jerry,
It still doesn’t work for me.

hello all,

it looks some developers could workaround this by increasing instantaneous OC limit, from 3.6A to 5A;
suggest let’s having another new discussion thread to be created for issue tracking.
we’ll need to gather more details, and also check why it doesn’t works for all scenario.
thanks

Ran into this issue running yolov4–with stock Jetpack 4.5.1 (3600 mA limit) running yolov4-tiny would trip the alert. Once I raised the limit 5000 mA, I could run yolov4-tiny in 15W 6-Core with no alert, but running full-blown yolov4 would still trip it.

How can I do that please?

hi all,

please check this discussion thread,

it’s confirmed this failure could be workaround by increased instantaneous OC limit from 3.6A to 5A.
however, it still able to reproduced with heavy GPU load even with 5A configuration.
let’s follow-up Topic 173571 for the use-case with heavy GPU load.
thanks

Ive successfully implemented this. Not been able to run a test yet but fingers crossed. Will this method last after reboot? If not, what can I paste and where to make it persistent?

Hi @mehmetdeniz. Do you know if your settings will persist after reboot. I haven’t been able to test it yet since implementing your solution as some of my team are using the NX for other tasks.

Hi @alex15, I tried to apply that method and temporally the error message disappeared. However the setting doesn’t persist on my jetson xavier nx and the problem appear after each reboot.
If anyone has a solution please answer me.

I Confirm that the workaround survives a reeboot by adding:
echo -n "5000" > /sys/devices/c250000.i2c/i2c-7/7-0040/iio\:device0/crit_current_limit_0
into /etc/systemd/nv.sh file.
Thanks!

2 Likes