关于nvidia启动时报L4TLauncher Access Denied的问题

我们同一批的22块Nvidia jetson Orin Nx R35.4.1单板中,有4块先后出现了一个很奇怪的问题。单板连续运行一段时间(大约1个月)后,突然无法正常SSH远程登陆,可以连接和输入密码,但反馈信息是“unable to cd to ‘/home/xt’”,随之连接自动断开。单板复位后,情况变得更糟糕了,无法正常进入系统。报错信息为

ESC to enter Setup.
F11 to enter Boot Manager Menu.
Enter to continue boot.
** WARNING: Test Key is used. **

L4TLauncher: Attempting Direct Boot
OpenAndReadUntrustedFileToBuffer: Failed to open boot\extlinux\extlinux.conf: Ac
cess Denied
ProcessExtLinuxConfig:sds Failed to Authenticate boot\extlinux\extlinux.conf (Ac
cess Denied)
L4TLauncher: Unable to process extlinux config: Access Denied
L4TLauncher: Attempting Kernel Boot
Android image header not seen
Failed to boot kernel:0 partition

我们进行了交叉比对,发现这个时候问题可能在西数(WD)的nvme SSD SN740硬盘上,因为将出现问题单板上的硬盘插到正常单板上,正常单板也出现同样问题。而且这些出错硬盘作为从盘挂载到linux主机中或者在nvidia的recovery系统中都可以正常挂载、读取、创建和删除文件。我们也对硬盘进行了硬件(smartctl)和文件系统(fsck,gdisk)的校验,均未发现问题。因此,我们进行了下述操作:

  1. 将一块出错硬盘的前100M空间通过dd命令清除,L4TLauncher不再报Access Denied的错误,当然也此时无法正常进入系统
  2. 将这块清除过前100M空间的硬盘插入单板,重新通过nvsdkmanager_flash.sh --storage nvme0n1p1烧写后,单板可以正常启动

因此,我们直接读取了正常和异常单板的前100M空间,如good.hex和bad.hex文件所示,的确存在较大的不同,可以看到,正常硬盘有一段安卓分区引导段,且异常硬盘的0x5000~0x1fea720和正常硬盘的0x5800~0x1feaf20数据是一致的。

但是异常硬盘为什么会出现这种变化呢。我曾经把异常硬盘的0x5000~0x1fea720数据后移与正常硬盘保持一致,但未解决问题。能否给予指导,谢谢

补充说明一下,我们今天检查发现,android引导头并不是引起问题的关键。我们存在没有这个android引导头的正常启动硬盘。两块硬盘的前100M空间,从0x5000开始往后全部一致。只在0x5000往前存在差别。我又做了一个尝试处理:

  1. 将一个正常硬盘的前100M空间复制到异常硬盘的前100M——尝试插入单板,启动失败,问题一致
  2. 将上述那块硬盘挂接进linux PC,执行sgdisk -G重做签名——尝试插入单板,启动失败,问题一致

我也尝试了将两块正常单板上的硬盘交换互插,结果两块单板也都能正常启动。但是插入出错的硬盘,立刻出现相同的问题

Hi,
Do you have Samsung NVMe SSD 980 or 990 for a try? Would like to ocnfim if it occurs on other vendor’s NVME SSD also.

Sorry, there are currently no NVMe SSDs from different manufacturers available on this type of board. However, I have seen posts with similar issues before, and they were using MMC, which also seemed to have the same problem.Issue about extlinux.conf: Access Denied - #10 by KevinFFF

Hi ylssrb,

The issue discussed in the thread you linked occurs intermittently, and the root cause has not yet been identified.

From your description and findings, this looks more like a problem with the SN740 SSD itself, so please check its firmware version or try a different NVMe to confirm.

You may also check if exlinux.conf can be read from UEFI shell when you hit the issue.

Shell> map -r
Shell> fs0:
fs0:\> dir \boot\extlinux\
fs0:\> type \boot\extlinux\extlinux.conf

long_text_64D30354-EC53-49D2-AA3C-10B109B38343.txt (16.7 KB)
The execution result of UEFI shell

new 90.txt (1.6 KB)
our rootfs may in fs2

FS2:\> type \boot\extlinux\extlinux.conf
type: Cannot open file - '\boot\extlinux\extlinux.conf'

The result you shared matches what you hit during boot.

Yes, you need to check the fs2 for rootfs in your case.
Please run the following commands to check again.

FS2:\> dir \boot\extlinux*
FS2:\> cd \boot\extlinux
FS2:\boot\extlinux> dir
FS2:\boot\extlinux> type extlinux.conf

A strange result shows that extlinux is a directory but cannot be entered by ‘cd’
long_text_B5258516-46D0-4DBD-B85C-18F5827E0687.txt (9.9 KB)

By the way, I just tried again and found that all the second-level subdirectories under the root file system are all inaccessible, and the reported errors are all “cd: ‘FS2:\xxxx\yyyy’ is not a directory.”

From your UEFI shell results, the rootfs on FS2 is visible, but the \boot\extlinux directory cannot be entered and extlinux.conf cannot be opened.
This indicates that the issue is on the SSD side or in the filesystem metadata as seen by UEFI, rather than on the Jetson board itself, since Jetson UEFI/L4T normally reads \boot\extlinux\extlinux.conf during boot and similar failures show up as L4TLauncher: Unable to process extlinux config: Access Denied.

Please get a different NVMe SSD to verify, if another SSD boots normally, that would further indicate the issue is with this SSD rather than the Jetson board.

Yes, our suspicion about the root cause of the issue aligns with yours. However, firstly, the storage device described in another issue is an MMC, which also experienced the same problem. Secondly, we have tried various methods to recover the hard disk, but none of them worked. Therefore, could you provide some assistance and inform us of what Jetson UEFI/L4T detects before reading \boot\extlinux\extlinux.conf, so that we can take more targeted actions.

I agree this should not be treated as an NVMe-only issue yet as there’s the similar issue found on AGX Orin with eMMC.
Before reading \boot\extlinux\extlinux.conf, Jetson UEFI/L4T first determines the boot mode and target rootfs slot, then accesses the rootfs filesystem, so your result suggests the failure may already happen at the filesystem/directory access stage before extlinux.conf itself is parsed.
Could you share the full serial console log with UEFI debug messages enabled to find if there’re other errors?

In the meantime, we still suggest verifying with different NVMe SSD to clarify the issue.

Well, we have purchased NVMe SSDs from different manufacturers for testing.
Also, are you referring to this log
MobaXterm_COM3USB-SERIALCH340COM3_20260630_155912.txt (38.8 KB)

  Jetson UEFI firmware (version 4.1-33958178 built on 2023-08-01T19:34:02+00:00)

This is the serial console log of r35.4.1 w/o debug message enabled.
You need to refer to Build with docker · NVIDIA/edk2-nvidia Wiki to build debug UEFI binary and update it so that there will be more debug messages output in UEFI to debug further.

In addition, r35.4.1 is such an old release.
You can also verify with the latest r35.6.4 or r36.5 or r39.2 for Orin NX.

Well, I have followed your advice and downloaded and compiled UEFI r35.4.1.
However, possibly due to the old version, there are some differences between the execution command and the description provided in the Post “Build with Docker”:
download command:edk2_docker edkrepo clone nvidia-uefi NVIDIA-Platforms uefi-202308.1
build command: edk2_docker edk2_nvidia/Platform/NVIDIA/Jetson/build.sh --init-deconfig ./edk2_nvidia/Platform/NVIDIA/Jetson/Jetson.defconfig
I modified the build command because I couldn’t find the Tegra directory under the edk2-nvidia/Platform/NVIDIA/ directory. Is this approach appropriate? Is the output a debug version?

You can simply run the following command to download the UEFI source for r35.4.1.

edk2_docker edkrepo clone nvidia-uefi-r35.4.1 NVIDIA-Platforms r35.4.1

And, run the following command to build UEFI binary.

edk2_docker edk2-nvidia/Platform/NVIDIA/Jetson/build.sh 

The debug UEFI binary could be found in images/uefi_Jetson_DEBUG.bin.
Please replace <Linux_for_Tegra>/bootloader/uefi_jetson.bin with the debug UEFI binary and perform the full flash to apply the change.

MobaXterm_COM4USB-SERIALCH340COM4_20260703_165814.txt (96.4 KB)
I have flashed the uefi_Jetson_DEBUG.bin that I built yesterday, and the serial console log is shown above. Please help confirm if it is helpful for solving the problem.

I have reviewed the debug UEFI log and do not see evidence that the NVMe is missing from PCIe or BootOrder.
The SSD is detected and its GPT/filesystem mappings are present, but the failure still happens exactly when L4TLauncher tries to open boot\extlinux\extlinux.conf, which suggests the issue is more likely at the filesystem/directory-access or metadata level seen by UEFI, rather than a simple device-detection problem.

In addition, do you enable bootloader secureboot or UEFI secureboot in your case?

Are you referring to this?

We should never have enabled it