AGX Orin SOM Custom Board Boot Time Issue (R35.5.0)

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Orin
, ,
1

Hello Team,

We are using the AGX Orin SOM with our custom carrier board running Jetson Linux R35.5.0. As previously discussed in this post: AGX Orin Custom Carrier Board Boot Time Issue, we are facing a long boot time issue.

Our custom board (0004) takes almost 2 minutes and 43 seconds. to boot, while the same code flashed on the EV kit (0000) boots in less than a minute.
Custom board boot log.txt (138.7 KB)
EVK boot log.txt (124.6 KB)

I’ve attached both boot logs with timestamps for your reference.

Could you provide any guidance on why the custom board is experiencing this delay and how to resolve it?

Best regards,
Parashuram

2

Hi,
Please check the quick start in developer guide and make sure you follow the steps one by one:
https://docs.nvidia.com/jetson/archives/r36.3/DeveloperGuide/IN/QuickStart.html
If the device still cannot be flashed/booted, please refer to the page to get uart log from the device:
https://elinux.org/Jetson/General_debug
If you are using custom board, you can compare uart log of developer kit and custom board to get more information.

Thanks!

3

Yes, I have followed the quick start guide and flashed the system as instructed, which is why I attached the logs.

The custom board is taking a long time to boot, as shown in the log excerpts below:

Custom Board Log:

[2024-10-11 17:27:52] E/LD:   init_elf:486 sys_open_ta_bin(bc50d971-d4c9-42c4-82cb-343fb7f37896)
[2024-10-11 17:27:52] E/TC:?? 00 ldelf_init_with_ldelf:131 ldelf failed with res: 0xffff000c

[2024-10-11 17:29:18] ESC   to enter Setup.
[2024-10-11 17:29:18] F11   to enter Boot Manager Menu.
[2024-10-11 17:29:18] Enter to continue boot.

EV Kit Log:

[2024-10-15 19:09:44] E/LD:   init_elf:486 sys_open_ta_bin(bc50d971-d4c9-42c4-82cb-343fb7f37896)
[2024-10-15 19:09:44] E/TC:?? 00 ldelf_init_with_ldelf:131 ldelf failed with res: 0xffff000c

[2024-10-15 19:09:52] ESC   to enter Setup.
[2024-10-15 19:09:52] F11   to enter Boot Manager Menu.
[2024-10-15 19:09:52] Enter to continue boot.

As you can see, even without modifying any code, the custom board takes significantly longer to boot than the EV kit, despite both using the same UEFI version:

Custom Board UEFI Version:

Jetson UEFI firmware (version 5.0-35550185 built on 2024-02-20T04:21:22+00:00)

EV Kit UEFI Version:

Jetson UEFI firmware (version 5.0-35550185 built on 2024-02-20T04:21:22+00:00)

I would appreciate any insights you may have regarding this issue.

Best regards,
Parashuram

5

Hi Parashuram.Biradar,

You have to enable UEFI debug message for the details of the delay on your custom carrier board.

Please also refer to Boot Time Optimization — NVIDIA Jetson Linux Developer Guide 1 documentation for boot time optimization on R35.5.0.

6

Thanks for your reply.

I cloned the edkrepo, removed the .inf file as mentioned in the wiki, generated the uefi_Jetson_DEBUG.bin, copied it to the bootloader path, and re-flashed the board. However, it still takes a long time to boot.

Here is the boot delay log for your reference:

[2024-10-16 19:17:09] add-symbol-file /home/parashuram/uefi_code/nvidia-uefi-r35.5.0/Build/Jetson/DEBUG_GCC5/AARCH64/Silicon/NVIDIA/Drivers/AndroidBootDxe/AndroidBootDxe/DEBUG/AndroidBootDxe.efi
[2024-10-16 19:17:09] Loading driver at 0x0082615D000 EntryPoint=0x008261643A8 AndroidBootDxe.efi
[2024-10-16 19:17:09] 
[2024-10-16 19:17:09] add-symbol-file /home/parashuram/uefi_code/nvidia-uefi-r35.5.0/Build/Jetson/DEBUG_GCC5/AARCH64/Silicon/NVIDIA/Drivers/PcieControllerDxe/PcieControllerDxe/DEBUG/PcieControllerDxe.efi
[2024-10-16 19:17:09] Loading driver at 0x0082614A000 EntryPoint=0x008261561B4 PcieControllerDxe.efi
[2024-10-16 19:17:09] 
[2024-10-16 19:17:10] add-symbol-file /home/parashuram/uefi_code/nvidia-uefi-r35.5.0/Build/Jetson/DEBUG_GCC5/AARCH64/SecurityPkg/Hash2DxeCrypto/Hash2DxeCrypto/DEBUG/Hash2DxeCrypto.efi
[2024-10-16 19:17:10] Loading driver at 0x0082613F000 EntryPoint=0x00826144D58 Hash2DxeCrypto.efi
[2024-10-16 19:17:10] 
[2024-10-16 19:17:10] add-symbol-file /home/parashuram/uefi_code/nvidia-uefi-r35.5.0/Build/Jetson/DEBUG_GCC5/AARCH64/Silicon/NVIDIA/Drivers/EqosDeviceDxe/EqosDeviceDxe/DEBUG/EqosDeviceDxe.efi
[2024-10-16 19:17:10] Loading driver at 0x008260A5000 EntryPoint=0x008260AC4A8 EqosDeviceDxe.efi
[2024-10-16 19:17:10] 
[2024-10-16 19:17:37] SNP:PHY: PhyDxeInitialization () Failed to reset Phy
[2024-10-16 19:17:37] DeviceDiscoveryBindingStart, driver returned Device Error to start notification
[2024-10-16 19:18:04] SNP:PHY: PhyDxeInitialization () Failed to reset Phy
[2024-10-16 19:18:04] DeviceDiscoveryBindingStart, driver returned Device Error to start notification
[2024-10-16 19:18:04] add-symbol-file /home/parashuram/uefi_code/nvidia-uefi-r35.5.0/Build/Jetson/DEBUG_GCC5/AARCH64/Silicon/NVIDIA/Drivers/NonDiscoverablePciDeviceDxe/NonDiscoverablePciDeviceDxe/DEBUG/NonDiscoverablePciDeviceDxe.efi
[2024-10-16 19:18:04] Loading driver at 0x0082609A000 EntryPoint=0x008260A08CC NonDiscoverablePciDeviceDxe.efi

Even after removing # Silicon/NVIDIA/Drivers/EqosDeviceDxe/EqosDeviceDxe.inf, the delay remains the same.
custom_boot.log (277.5 KB)

I would appreciate any further insights you may have regarding this issue.

Regards,
Parashuram

7

The system is now working fine with reduced boot time. Thank you for your support.

I would like to know how to disable the UEFI boot log from being displayed during boot using the custom compiled edk2-nvidia source. I no longer need the UEFI logs to appear during boot.

Does removing the EqosDeviceDxe.inf affect any other part of the code, such as the 1G network or any other network ports?

Regards,
Parashuram

8

After removing the EqosDeviceDxe.inf in UEFI, the 1GbE MDI network is not working, whereas it was working before the UEFI changes.

Attached is the dmesg log for reference.

[   10.941126] nvethernet 2310000.ethernet: Adding to iommu group 51
[   10.947985] nvethernet 2310000.ethernet: failed to read skip mac reset flag, default 0
[   10.956642] nvethernet 2310000.ethernet: failed to read MDIO address
[   10.963659] nvethernet 2310000.ethernet: setting to default DMA bit mask
[   10.971029] nvethernet 2310000.ethernet: max-platform-mtu DT entry missing, setting default 1500
[   10.980603] nvethernet 2310000.ethernet: missing nvidia,pad_auto_cal_pu_offset, setting default 0
[   10.990208] nvethernet 2310000.ethernet: missing nvidia,pad_auto_cal_pd_offset, setting default 0
[   11.255735] nvethernet 2310000.ethernet: No MAC address in local DT!
[   11.262771] nvethernet 2310000.ethernet: failed to get MAC address
[   11.272290] nvethernet: probe of 2310000.ethernet failed with error -22

Regards,
Parashuram

9

Yes, EqosDeviceDxe may affect the function of ethernet.
There are still some methods in the guide I shared to reduce the boot time.
Have you also tried each of them?

You can just use uefi_Jetson_RELEASE.bin instead of uefi_Jetson_DEBUG.bin if you don’t need the debug messages.

10

Is there any solution to resolve this issue? I’d like to understand what’s causing the delay and what is happening with the EqosDeviceDxe that affects the 1G Ethernet functionality. Is there a timeout function or any other factor causing the delay, and if so, how can we address it?

Apart from the methods mentioned in the wiki, is it possible to reduce the boot time without impacting the 1G Ethernet?

Regards,
Parashuram

11

Please help to add more logs in the eqos driver and let us know the exact line causing your delay since we don’t see that on the devkit.

Yes, there’re other method mentioned in the guide, please refer to them.
e.g. have you removed the 5s autoboot timeout in UEFI?

12

We are working on it. I will share the update.

15

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.