Orin won't boot after successive reboots

Hello NVIDIA engineers:
Orin won’t boot after successive reboots

I have a problem with the Jetson agx orin (Jetson R35.2.1):
After executing “sudo reboot” three times in a row in the uart terminal, the rootfs fails to boot with the following error log:

I have to run flash.sh again to burn the rootfs, but the problem is still reproduced after three consecutive “sudo reboots”.

  1. what is the reason for this problem and why this mechanism exists?
  2. how to disable this mechanism?
  3. If I can’t disable this mechanism, how can I exit bash-5.0 to boot normally?

Hi hanyu.du,

Are you using the devkit or custom board for AGX Orin?
Could you help to verify with latest R35.3.1?

Do you run this command manually after boot up?

Could you share the full flash log for further check?

Please also make sure you run the sudo reboot after nv-l4t-bootloader-config.service started:

$ sudo systemctl status nv-l4t-bootloader-config

This seems to be the same problem as the one I encountered in this post

We used R35.3.1 and R35.2.1 and did not find that sudo reboot would detect this situation when the device was not powered on.But why do we find this situation when using R35.3.1 and R35.2.1 on our custom devices? Can we turn it off without the need for a protective mechanism?

This is very important for our use. Could you please help us analyze how to avoid it.

Sorry, this is failsafe mechanism. You could just configure its counts through adding ROOTFS_RETRY_COUNT_MAX=X and it could not be disabled.

I think the issue is caused from nv-l4t-bootloader-config.service started after local.rc running at boot up.

Hello Kevin,
Sorry for the late reply, I’ve been doing some testing on both devkit and custom board for the past few days.
Found some problems:

  1. testing NVIDIA stock Linux_for_Tegra does not go into recovery boot, sudo systemctl status nv-l4t-bootloader-config prints:

  2. testing our customized Linux_for_Tegra causes stuck in recovery boot issues.
    sudo systemctl status nv-l4t-bootloader-config prints:

The attachment uart_log.txt is a printout of the last “sudo reboot” that resulted in a recovery boot.
uart_log.txt (87.2 KB)

Let’s try to configure sudo ROOTFS_RETRY_COUNT_MAX=3 . /flash.sh xxx but it didn’t do anything.
Can you give us some advice on how to troubleshoot this, or is there a way to disable the Recovery boot mechanism.

It is a one time service, and it seems the status=0 and SUCCESS in both cases.

Could you help to run the reboot test as following steps to verify on your custom board?

1. power on the target(this is a cold boot, this is must)
2. check the background service status:
$ sudo systemctl status nv-l4t-bootloader-config
3. do 1st reboot target only when the service status is SUCCESS
4. login in target, and check the background service status,
5. do 2nd reboot target only when the service status is SUCCESS
6. login in target, and check the background service status,
7. do 3rd reboot target only when the service status is SUCCESS
8. login in target, and check the background service status,
9. do 4th reboot target only when the service status is SUCCESS
..

(i.e. make sure the background service status is SUCCESS before rebooting the target.)

Thanks for your suggestion.

I was testing the official NVIDIA image and found that the nv-l4t-bootloader-config service is SUCCESS immediately after booting.

I was testing our custom image and noticed that the nv-l4t-bootloader-config service seems to become SUCCESS after some time after booting (maybe 1-2 minutes after booting).

Comparing with the official NVIDIA image I only modified the /boot/dtb/kernel_tegra234-p3701-0004-03737-0000.dtb file. Can modifying the device tree cause the service to start slowly?
Do you have any idea what this might be due to?

Do you modify or add any system service to cause it started slowly?
Or modify any boot up script like you add reboot command in rc.local?

Is there any messages on your serial console log during this period?

Hello Kevin,
No boot self-start services such as rc.local were being used.
I found where the device tree problem was.

It’s in sources/hardware/nvidia/platform/t23x/concord/kernel-dts/cvb/tegra234-p3737-0000-a04.dtsi. when I change here to ‘disable’, it gives the nv-l4t-bootloader -config service starts slowly. After I modify it to ‘okay’, the service can start directly after booting again, but a new problem occurs, the kernel prints the error log:
uart_log_error.txt (110.4 KB)


After I changed the device tree to ‘disable’, rebooting again does not print the error log, but there is a problem with the nv-l4t-bootloader-config service starting slowly after booting.

It seems the timeout error causing from usb.
Do you have any custom design in USB different from the devkit?

hello,
Have not customized the usb.
I don’t have a clue about this kernrl print error at the moment, investigating the previous issue might be a breakthrough.

Resolve slow loading of nv-l4t-bootloader-config service.

Can you help to find out why the slow loading of the nv-l4t-bootloader-config service occurs after modifying the “status of tegra_xudc = “disabled”;”? I was able to reproduce the slow loading of the nv-l4t-bootloader-config service by using devkit + the official NVIDIA L4T image and then modifying the NVIDIA default device tree file: ‘status’.

tegra_xudc is a device driver for USB.
You could refer to the following instructions for detail.
Jetson AGX Orin Platform Adaptation and Bring-Up — Under the usci_ccg Node

The nv-l4t-bootloader-config.service is configured after few services, and one of them is relating to USB.

Why would you disable this node(tegra_xudc) for your custom board?

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.