@sam.tsai were you able to solve this issue ?
I am also working on YOLOV4 with heavy usage, and i get system throttled message on JetPack 4.5.1, but not on JetPack 4.4.
Hi,
I still canāt solve this issue on JetPack 4.5, as you said, it is pass on Jetpack 4.4.
Nvidia did not give a truly effective solution on JetPack 4.5, so I will skip this issue for first.
Need to make some clarifications for this issue.
-
The OC throttling case also appears in jetpack4.4.1. The real cause that makes you feel this issue does not happen on jp4.4.1 is because there was no āOC alarm notificationā mechanism implemented in jp4.4.1. Thus, jp4.5 and jp4.4.1 actually both have oc throtting.
-
The oc throtting is due the the hardware limitation of NX module. We currently set OC limit to 3.6A to protect the NX hardware.
-
The suggestion to prevent the OC throtting notification are:
- Set the OC limit to 5A. This shall still improve the performance and still not do harm to the module.
- Use the power estimator to adjust the nvpmodel conf. The throttling may not happen, but the performance may drop.
1 Like
Hey thanks, i have yet to try to power estimator, will get back after that.
- āThe real cause that makes you feel this issue does not happen on jp4.4.1 is because there was no āOC alarm notificationā mechanism implemented in jp4.4.1ā - yes okay, that definitely makes sense then.
My issue is that in JP4.4 running deepstream YOLOV4 model, i was able to get 16FPS on both camera streams of 4K without any issues, but on JP4.5.1 one of the camera just goes blank and the other camera source is about 8 FPS. this happened even after setting the OC limit.
This is on JP4.5.1
Iāve checked the same configuration file works on JP4.4, so now i am not sure what could be the problem.
I canāt share the config file because we are using a custom plugin which also does a lot of processing.
On your jetpack 4.4.1, please check
/sys/kernel/debug/bpmp/debug/soctherm/oc# cat oc_1/event_cnt
/sys/kernel/debug/bpmp/debug/soctherm/oc# cat oc_1/irq_cnt
This will record the oc event. We can compare the event cnt between jp4.4.1 and jp4.5.1 with same application running.
1 Like
Hi, we did some testing, The following numbers are after running our application for sometime, ~10 mins
On JP 4.5.1
$ sudo cat /sys/kernel/debug/bpmp/debug/soctherm/oc/oc_1/event_cnt
>> 3099
$ sudo cat /sys/kernel/debug/bpmp/debug/soctherm/oc/oc_1/irq_cnt
>> 1548
On JP 4.4
$ sudo cat /sys/kernel/debug/bpmp/debug/soctherm/oc/oc_1/event_cnt
>> 3601
$ sudo cat /sys/kernel/debug/bpmp/debug/soctherm/oc/oc_1/irq_cnt
>> 1780
I have some good news, I set the OC limit to 7000 mA instead of 5000mA on JP 4.5.1, and the warning disappeared ! i was a little worried, since i am getting it out of specs, so we tested it for an hour, with no issues, we observed better sync between the frames of the two cameras as well !
And no more issues with the camera as well ! it turns out i had set sync-inputs=true for the nvstreammux which caused one of the camera to black out, it was never the power issue, this sync-inputs thing is only there on JP 4.5.1.
I believe there are some issue with synchronizing frames between two cameras in nvstreammux using sync-inputs=true, expected behaviour should be despite different fps of cameras, it should still be able to synchronize the two cameras.
Just to add another report:
I am using a pose estimation network in OpenCV (compiled with CUDA support) decoding a RTSP video stream (h264 decoding with hardware accelerated ffmpeg backend). CPU is at about medium load.
I applied the OC limit setting for 5000 ma. In 15W / 6 cores power mode I get the over-current throttling warning as well. Attached to the board is a RasPi HQ camera bit it is not used in that case. A monitor is connected to the HDMI port.
Hi @helge ,
Can you file your own topic and paste the your power estimator result so that we can check what is going on?