When flashing Jetson AGX Orin to NVMe using l4t_initrd_flash.sh, the device fails to boot with "mounting PARTUUID=xxx as /mnt fail" error

Robotics & Edge Computing Jetson Systems Jetson AGX Orin
1

Environment:

  • Hardware: Jetson AGX Orin Developer Kit
  • L4T Version: R36.4.3
  • Flash Method: l4t_initrd_flash.sh with external NVMe

Problem:
After successful flash completion, device fails to boot from NVMe. Serial console shows:

[ 29.328034] ERROR: mounting PARTUUID=4934d836-4f19-4da5-8758-f88b3c9e088d as /mnt fail…
[ 29.329409] ERROR: PARTUUID=4934d836-4f19-4da5-8758-f88b3c9e088d mount fail…

flashing command:

sudo ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 -c tools/kernel_flash/flash_l4t_t234_nvme.xml --showlogs --network usb0 jetson-agx-orin-devkit external

flash finish log:

Writing qspi_bootblob_ver.txt (parittion: A_VER) into /dev/mtd0
Sha1 checksum matched for /mnt/internal/qspi_bootblob_ver.txt
Writing /mnt/internal/qspi_bootblob_ver.txt (109 bytes) into /dev/mtd0:66977792
Copied 109 bytes from /mnt/internal/qspi_bootblob_ver.txt to address 0x03fe0000 in flash
Writing gpt_secondary_3_0.bin (parittion: secondary_gpt) into /dev/mtd0
Sha1 checksum matched for /mnt/internal/gpt_secondary_3_0.bin
Writing /mnt/internal/gpt_secondary_3_0.bin (16896 bytes) into /dev/mtd0:67091968
Copied 16896 bytes from /mnt/internal/gpt_secondary_3_0.bin to address 0x03ffbe00 in flash
[ 226]: l4t_flash_from_kernel: Successfully flash the qspi

writing item=17, 9:0:secondary_gpt, 61203267072, 16896, gpt_secondary_9_0.bin, 16896, fixed–0, ef3783b4a8758c0ba89a9927ee6a3271d7558ad0
[ 655]: l4t_flash_from_kernel: Successfully flash the external device
[ 655]: l4t_flash_from_kernel: Flashing success
[ 655]: l4t_flash_from_kernel: The device size indicated in the partition layout xml is smaller than the actual size. This utility will try to fix the GPT.
[ 655]: l4t_flash_from_kernel: The device size indicated in the partition layout xml is smaller than the actual size. This utility will try to fix the GPT.
Flash is successful
Reboot device
Cleaning up…
Log is saved to Linux_for_Tegra/initrdlog/flash_3-6.2_0_20260603-103800.log

3

*** Please note that this reply is generated by LLM automatically ***

The issue you’re experiencing with the Jetson AGX Orin Developer Kit failing to boot from an NVMe device after a successful flash using l4t_initrd_flash.sh is likely related to the partition configuration or the mounting process.

First, let’s break down the error messages you’re seeing:

  1. 29328034 ERROR mounting PARTUUID=4934d8364f194da58758f88b3c9e088d as mnt fail
  2. 29329409 ERROR PARTUUID=4934d8364f194da58758f88b3c9e088d mount fail

These errors indicate that the system is unable to mount the partition with the specified PARTUUID. This could be due to a variety of reasons, such as:

  • The partition is not properly formatted or configured.
  • The PARTUUID is incorrect or not recognized by the system.
  • There are issues with the NVMe device or its connection.

To troubleshoot this issue, you can try the following steps:

  1. Verify the partition configuration: Check the partition table of the NVMe device to ensure that it is properly configured and formatted. You can use tools like fdisk or gdisk to inspect the partition table.
  2. Check the PARTUUID: Verify that the PARTUUID specified in the error message matches the actual PARTUUID of the partition. You can use tools like blkid to retrieve the PARTUUID of the partition.
  3. Inspect the system logs: Check the system logs for any additional error messages or clues that may indicate the cause of the issue.
  4. Try a different flash method: If you’re using l4t_initrd_flash.sh with the --external-device option, try using the flash.sh script instead. This may help to rule out any issues with the l4t_initrd_flash.sh script.
  5. Check the NVMe device: Verify that the NVMe device is properly connected and recognized by the system. You can use tools like lspci or nvme list to inspect the NVMe device.

In comparison to other Jetson modules, the Jetson AGX Orin Developer Kit has a similar partition configuration to the Jetson Orin NX and Jetson Orin Nano modules. However, the flash method and partition configuration may differ depending on the specific module and use case.

For example, the Jetson Orin NX module uses a similar partition configuration to the Jetson AGX Orin module, but with some differences in the partition layout and sizes. The Jetson Orin Nano module, on the other hand, uses a different partition configuration that is optimized for its smaller storage capacity.

In terms of troubleshooting, the steps outlined above can be applied to other Jetson modules as well, with some modifications to account for the specific partition configuration and flash method used.

Here are some additional resources that may be helpful in troubleshooting this issue:

  • The Jetson AGX Orin Developer Kit documentation provides detailed information on the partition configuration and flash method.
  • The Jetson Orin NX and Jetson Orin Nano documentation provide information on the partition configuration and flash method for those modules.
  • The NVIDIA Jetson forum provides a wealth of information and resources for troubleshooting and resolving issues with Jetson modules.

By following these steps and resources, you should be able to troubleshoot and resolve the issue with the Jetson AGX Orin Developer Kit failing to boot from an NVMe device.

*** Please give the thumbs up if you get answers you like. Or provide feedback to help us improve the answer. ***

4

bash-5.1# blkid /dev/nvme0n1*
/dev/nvme0n1: PTUUID=“1b042d03-1d9a-49d7-b8c2-097cc3f5120f” PTTYPE=“gpt”
/dev/nvme0n1p1: UUID=“a1b63796-1688-4d3a-b92c-f59f35915b53” BLOCK_SIZE=“4096” TYPE=“ext4” PARTLABEL=“APP” PARTUUID=“c3ecd21f-888a-4e24-9f23-9b17cd63b596”
/dev/nvme0n1p10: UUID=“3303-17D2” BLOCK_SIZE=“512” TYPE=“vfat” PARTLABEL=“esp” PARTUUID=“1a6a4552-1ede-48d6-89ec-274efd3d697c”
/dev/nvme0n1p11: PARTLABEL=“recovery_alt” PARTUUID=“21ba6b3b-d724-4e87-a12a-5626bf9b9e00”
/dev/nvme0n1p12: PARTLABEL=“recovery-dtb_alt” PARTUUID=“07926a65-41c8-422e-ad38-051bd7077a7c”
/dev/nvme0n1p13: PARTLABEL=“esp_alt” PARTUUID=“02605f9c-bda8-401e-842f-6f3b38c4d16f”
/dev/nvme0n1p14: PARTLABEL=“UDA” PARTUUID=“7d7467a6-e1a6-4ce9-ab24-9e6c57a44c02”
/dev/nvme0n1p15: PARTLABEL=“reserved” PARTUUID=“44305aa4-8a9b-410a-8908-c07358b80a40”
/dev/nvme0n1p2: PARTLABEL=“A_kernel” PARTUUID=“6e0c2257-60f4-4103-ba68-282129cfcc6f”
/dev/nvme0n1p3: PARTLABEL=“A_kernel-dtb” PARTUUID=“61fef8aa-4c5d-48e4-9bea-5f4064ecec29”
/dev/nvme0n1p4: PARTLABEL=“A_reserved_on_user” PARTUUID=“5ee43084-9341-490a-a717-d84b2e9d661a”
/dev/nvme0n1p5: PARTLABEL=“B_kernel” PARTUUID=“74e79d71-1dd4-4032-bb84-1935ad27f53e”
/dev/nvme0n1p6: PARTLABEL=“B_kernel-dtb” PARTUUID=“6d71649c-a9fd-4d55-a2b2-7a31f35fcc7c”
/dev/nvme0n1p7: PARTLABEL=“B_reserved_on_user” PARTUUID=“04d83cb1-78fe-4746-af66-6c7432647371”
/dev/nvme0n1p8: PARTLABEL=“recovery” PARTUUID=“7bda5db3-9741-4951-a8f1-e95674fe9a22”
/dev/nvme0n1p9: PARTLABEL=“recovery-dtb” PARTUUID=“704efac8-e45e-4966-a4b6-ad77201d5b5e”
bash-5.1# umount /mnt
bash-5.1# mount /dev/nvme0n1p1 /mnt
bash-5.1# [ 187.549778] EXT4-fs (nvme0n1p1): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.

bash-5.1#
bash-5.1# grep APPEND /mnt/boot/extlinux/extlinux.conf
APPEND ${cbootargs} root=PARTUUID=c3ecd21f-888a-4e24-9f23-9b17cd63b596 rw rootwait rootfstype=ext4 mminit_loglevel=4 console=ttyTCU0,115200 firmware_class.path=/etc/firmware fbcon=map:0 nospectre_bhb video=efifb:off console=tty0 iommu=pt usbc

5

Hi @xuqiang0521,

From your latest output, the APP partition on the NVMe is:

/dev/nvme0n1p1 PARTUUID="c3ecd21f-888a-4e24-9f23-9b17cd63b596"

and /mnt/boot/extlinux/extlinux.conf also uses that same PARTUUID.

However, the boot failure is trying to mount a different PARTUUID:

4934d836-4f19-4da5-8758-f88b3c9e088d

So the NVMe APP filesystem itself looks mountable, but the boot flow appears to be using a stale or different rootfs command line.

Could you please check the actual kernel command line from the failure shell?

cat /proc/cmdline
ls -l /dev/disk/by-partuuid

Also, please enter the UEFI boot menu and make sure the selected boot option is the NVMe device you just flashed. The R36.4.3 Developer Guide notes that after flashing to NVMe, the corresponding NVMe boot media should be selected from the UEFI menu.

Recommendation: please share the output of the two commands above first. If /proc/cmdline still contains root=PARTUUID=4934d836-..., then we need to find where that stale boot entry or command line is coming from.

Hope this helps!

6

We found that the UEFI was booting from eMMC every time; modifying the UEFI bin file solved the problem