AGX Xavier power cut suddenly

[stephen.chan]

Hi all,

We posted a topic before in AGX Xavier shutdown suddenly - #13 by ShaneCCC. However, we don’t have a solution yet.

After few weeks investigation, we believed that the power cut is related to power supply problem.
It is because the power cut immediately when we turned to MODE_30W_6CORE (We are running on DC). After replacing the cable with much thicker one, it can run at MODE_30W_6CORE more than 2 hours. We thought the problem is gone.

However, the Xavier power cut again after 48 hours at MODE_15W_DESKTOP.
My question are:

1. What’s the differences between MODE_15W and MODE_15W_DESKTOP?
2. We turned on the “Jetson Clock”, is this a good practice to lock the power consumption in our case?

Thanks for any advise

[KevinFFF]

Hi stephen.chan,

Are you using the devkit or custom board for AGX Xavier?
What’s you Jetpack version in use?

You could check /etc/nvpmodel.conf to know the configuration for different power mode.

When you run jetson_clocks, it would use the static maximum frequencies of the CPU, GPU and EMC clocks.
May I know what’s your use case? Do you run any applications on your board?

[stephen.chan]

Our program is to capture the traffic situation and some specific type of vehicles. It connects to a PoE camera via a ethernet switch. Eventually, based on a business case, the program exports a video from the camera and upload to data center.

We are suffering power sudden cut off problem. It happened occasionally, sometimes after 12 hours, sometimes 48 hours… no clue yet. We already replaced DC battery, cables, switch and Xavier too, but still no luck.

[KevinFFF]

@stephen.chan

We are suffering power sudden cut off problem.

Is there any error message in serial console log before the power cut off?
Have you also tried to use the devkit with the same setup to reproduce the issue?

[linuxdev]

I will add that Jetsons are quite sensitive to power regulation quality. Also, I am curious, what do you see from this command:

sudo -s
egrep -a -H '.*' `find /sys -name reason`
exit

(I’m using the sudo shell because it is one command embedded in another)

[stephen.chan]

Thanks Kevin, actually the machine is installed in a very remote area (outdoor), we can’t debug via the serial console log easily.
We installed 5 set equipment (Camera + Router + Xavier + Battery + Remote switch) in 5 different location. Only one of them is hitting this problem.

[KevinFFF]

@stephen.chan

we can’t debug via the serial console log easily.

You could also check if there’s any error messages in /var/log/kern.log or /var/log/syslog.

[stephen.chan]

[stephen.chan]

Any clue? I found the clock was out-sync for a while after restarting

[stephen.chan]

Nothing I can see.
What’s the purpose of this command? Are we going to search the pattern “.*” in the content of the files, which are named as “reason”?

[KevinFFF]

@stephen.chan

Any clue?

I would suggest comparing the normal working log and the log with power cut issue because you said only one setup having this issue.

[linuxdev]

The file “reason”, within “/sys”, shows a reason for the last reboot. I think the PMIC driver creates this file and dynamically inserts it into the device tree during boot based on the previous system off condition (e.g., software reset, power loss, so on). However, that was in L4T R32.x, and it looks like the PMIC reason file is no longer created (I was so used to having that I didn’t realize R35.x does not).

I guess I should ask the question to @KevinFFF, is there anything similar to the device tree reason file on R35.x?

[KevinFFF]

Hi linuxdev,

For rel-35, you could read reset cause in MB1 log as following.

[0000.064] I> MB1 (version: 1.2.0.0-t234-54845784-562369e5)
[0000.069] I> t234-A01-1-Silicon (0x12347) Prod
[0000.074] I> Boot-mode : Coldboot
[0000.077] I> Entry timestamp: 0x00000000
[0000.081] I> last_boot_error: 0x0
[0000.084] I> BR-BCT: preprod_dev_sign: 0
[0000.087] I> rst_source: 0xb, rst_level: 0x1

[linuxdev]

It would be useful if this were added to a “/sys” export. Then, if boot is not set for logging, the reason would still be available. Not terribly important, but useful.

[KevinFFF]

I’ve found there’s a node for reset reason in R35.

Take AGX Xavier as example:

$ cat /sys/devices/platform/c360000.pmc/reset_reason

[stephen.chan]

Thanks Kevin, this is very useful. Let me share the code when it happens again.

[stephen.chan]

@KevinFFF

cat /sys/devices/platform/c360000.pmc/reset_reason

It is SYS_RESET_N, power supply problem?

[Trumany]

Hi, please read the chapter DV/Dt Circuit Considerations in Xavier Design Guide. Your issue looks like be caused by Dv/Dt. You can try the rework listed in the chapter. Besides above, you also should do more tests to make sure if it is caused by poor power supply, like using other supplies and etc.