RAS Error in L2 and failed to boot after flashed R35.4.1 on Xavier NX

Robotics & Edge Computing Jetson & Embedded Systems Jetson Xavier NX
,
1

Hello, team, I am porting R35.4.1 on Jetson Xavier NX emmc module（with custom carrier board）, but after I flashed the module, it can not boot up successfully, and log as follows:

Jetson UEFI firmware (version 4.1-33958178 built on 2023-08-01T19:34:02+00:00)
ESC   to enter Setup.
F11   to enter Boot Manager Menu.
Enter to continue boot.
**  WARNING: Test Key is used.  **

L4TLauncher: Attempting Direct Boot
L4T boot options
0: nvme kernel
1: recovery kernel
Press 0-1 to boot selection within 3.0 seconds.
Press any other key to boot default (Option: 0)
EFI stub: Booting Linux Kernel...
EFI stub: Using DTB from configuration table
EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path
EFI stub: Exiting boot services and installing virtual address map...
ÿㄒROR:   MPIDR 0x80000000: exception reason=0 syndrome=0xbe000000
ERROR:   **************************************
ERROR:   RAS Error in L2, ERRSELR_EL1=0x200:
ERROR:   	Status = 0xfc00640d
ERROR:   	IERR = SCF to L2 Decode Error Read: 0x64
ERROR:   SERR = Illegal address (software fault): 0xd
ERROR:   	Overflow (there may be more errors) - Uncorrectable
ERROR:   	Uncorrectable (this is fatal)
ERROR:   	MISC0 = 0x80000000100000
ERROR:   	MISC1 = 0x20240000005
ERROR:   	ADDR = 0x800000000f40ffe8
ERROR:   **************************************
ERROR:   **************************************
ERROR:   RAS Error in SCF_IOB, ERRSELR_EL1=0x401:
ERROR:   	Status = 0xf4009604
ERROR:   	IERR = CBB Interface Error: 0x96
ERROR:   SERR = Assertion failure: 0x4
ERROR:   	Uncorrectable (this is fatal)
ERROR:   	MISC0 = 0x40
ERROR:   	MISC1 = 0x1664a644461
ERROR:   	ADDR = 0x800000000f40ffe8
ERROR:   **************************************
ERROR:   RAS error handled!
ERROR:   sdei_dispatch_event returned -1

Here is flash log:
flash_full_log.txt (64.8 KB)

Xavier NX log:
xavier_nx-r35.4.1-bootup.txt (24.8 KB)

Please help me to solve this ,thanks

2

I try to apply this ,but not worked.

# ls bootloader/tos_t194.img -l
lrwxrwxrwx 1 root root 21  8月 15 10:06 bootloader/tos_t194.img -> tos-mon-only_t194.img

# cat p3668.conf.common | grep tos
TOSFILE="bootloader/tos-mon-only_t194.img";
4

Hi @WakkeWang

Does this issue always happen? It sounds the reproduce rate is quite high.

5

It always happens.

6

It sounds not related to previous RAS error because previous is low reproduce rate. Does your custom board boot from USB dirve?

7

My carrier board and Xavier Nx works well on R35.3.1 and R35.2.1 after using your patch (tos-mon-only_t194.img) , but alaways failed on R35.4.1.

My board boot from EMMC, but it has a nvme hard drive on board.

8

  1. Could you fallback to r35.3.1 and do not use mon-only patch and see if you can still 100% see this issue?
    Because the mon-only patch is for an issue that reproduced in very low reproduce rate (1/800). An issue with 100% reproduce rate sounds totally not related to the mon-only patch.

  2. I guess the result will be good but still want to ask, are you able to reproduce this issue on NV devkit?

10

the xavier nx + Nv devkit can boot up successfully.
the xavier nx + my custom carrier board came the RAS error.

11

Have you checked the rel-35.3.1 with default tos image?

12

Do you mean I use tos-optee_t194.img on R35.3.1 ?

13

Yes, I want to know your default situation on rel-35.3.1.

14

OK. I will do the test and let you know.

15

R35.3.1 came the same RAS error


L4TLauncher: Attempting Direct Boot
L4T boot options
0: nvme kernel
1: recovery kernel
Press 0-1 to boot selection within 3.0 seconds.
Press any other key to boot default (Option: 0)
EFI stub: Booting Linux Kernel...
EFI stub: Using DTB from configuration table
EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path
EFI stub: Exiting boot services and installing virtual address map...
ÿㄒROR:   MPIDR 0x80000000: exception reason=0 syndrome=0xbe000000
ERROR:   **************************************
ERROR:   RAS Error in L2, ERRSELR_EL1=0x200:
ERROR:   	Status = 0xfc00640d
ERROR:   	IERR = SCF to L2 Decode Error Read: 0x64
ERROR:   SERR = Illegal address (software fault): 0xd
ERROR:   	Overflow (there may be more errors) - Uncorrectable
ERROR:   	Uncorrectable (this is fatal)
ERROR:   	MISC0 = 0x80000000100000
ERROR:   	MISC1 = 0x20240000000
ERROR:   	ADDR = 0x800000000f40ffe8
ERROR:   **************************************
ERROR:   **************************************
ERROR:   RAS Error in SCF_IOB, ERRSELR_EL1=0x401:
ERROR:   	Status = 0xf4009604
ERROR:   	IERR = CBB Interface Error: 0x96
ERROR:   SERR = Assertion failure: 0x4
ERROR:   	Uncorrectable (this is fatal)
ERROR:   	MISC0 = 0x40
ERROR:   	MISC1 = 0x264a644421
ERROR:   	ADDR = 0x800000000f40ffe8
ERROR:   **************************************
ERROR:   RAS error handled!
ERROR:   sdei_dispatch_event returned -1
16

please enable the UEFI debug log by rebuilding the UEFI binary.

Also, in the meantime, please check review your hardware design.

17

Hi , sorry to reply so late, how can I enable debug log on UEFI ?

18

Rebuild the UEFI by following this github.

There would be a debug binary built out. Replace that binary with the default one within your BSP.

19

Here is the bootup log when enable uefi debug:
uefi-debug-bootlog.txt (203.9 KB)

20

looks like this is already loading kernel.

Please add this to kernel cmdline and see if anything new got printed.

earlycon=uart8250,mmio32,0xc280000

Add it to extlinux.conf (/boot/extlinux/extlinux.conf) and reflash the board.

21

Hi I found that, when I disable NVME SSD boot and choose emmc boot on UEFI boot Menu , then it can successfully boot up. and I think that is why it can boot up successfully on NV DEV Kit carrier board without nvme ssd.

and why RAS error happend when the first boot order is nvme ssd ?

1 Like
22

Hi,

Do you need PCIe or NVMe in use on your custom board? I would like to make a test here to disable PCIe controller and see if this can bypass your error.