Correctly using l4t_initrd_flash to flash to external NVMe without sdkmanager

Autonomous Machines Jetson & Embedded Systems Jetson AGX Orin
,
1

Hi, I need some help. I’m trying to flash a Jetson AGX Orin Industrial on a CTI Rogue carrier board onto a 2TB NVMe that’s already attached to the carrier board. I have read through “Flashing with initrd” and I believe it is what I want to do. From there, I have read through README_initrd_flash.txt in Linux_for_Tegra/tools/kernel_flash, and determined that I am interested in Workflow 3: How to flash to an external storage. I’m trying to follow the instructions there as closely as I can, but that’s where I run into problems.

I have modified flash_l4t_external.xml such that the device type is set to “nvme” and the device num_sectors is set to 4000000000. (2.048x10^12 B / 512 B/sector = 4000000000 sectors). I saved this as a new file called test_flash_l4t_external.xml.

The command I’m running from the Linux_for_Tegra folder is

sudo ADDITIONAL_DTB_OVERLAY_OPT="BootOrderNvme.dtbo" ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1 -c ./tools/kernel_flash/test_flash_l4t_external.xml cti/orin-agxi/rogue-orin/base nvme0n1p1

I have also tried

sudo ADDITIONAL_DTB_OVERLAY_OPT="BootOrderNvme.dtbo" ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 -c ./tools/kernel_flash/test_flash_l4t_external.xml cti/orin-agxi/rogue-orin/base nvme0n1p1

because README_initird_flash.txt is unclear about which is preferred. In either case, I get to a point where it finishes all the stuff it does before actually flashing the device, and I see:

saving flash command in flashcmd.txt

*** no-flash flag enabled. Exiting now… ***

User can run above saved command in factory environment without
providing pkc and sbk keys to flash a device

Example:

$ cd bootloader 
$ sudo bash ./flashcmd.txt

But when I run sudo bash ./flashcmd.txt, it gives the following output.

 Entering RCM boot

[   0.0424 ] mb1_t234_prod_aligned_sigheader.bin.encrypt filename is from --mb1_bin
[   0.0424 ] psc_bl1_t234_prod_aligned_sigheader.bin.encrypt filename is from --psc_bl1_bin
[   0.0424 ] rcm boot with presigned binaries
[   0.0441 ] tegrarcm_v2 --new_session --chip 0x23 0 --uid --download bct_br br_bct_BR.bct --download mb1 mb1_t234_prod_aligned_sigheader.bin.encrypt --download psc_bl1 psc_bl1_t234_prod_aligned_sigheader.bin.encrypt --download bct_mb1 mb1_bct_MB1_sigheader.bct.encrypt
[   0.0457 ] BR_CID: 0x80012344705DD80B1800000011010080
[   0.0471 ] Sending bct_br
[   0.0474 ] Sending mb1
[   0.0475 ] Sending psc_bl1
[   0.0606 ] Sending bct_mb1
[   0.0684 ] Generating blob for T23x
[   0.0738 ] tegrahost_v2 --chip 0x23 0 --generateblob blob.xml blob.bin
[   0.0754 ] The number of images in blob is 19
[   0.0759 ] blobsize is 68123211
[   0.0760 ] Added binary blob_uefi_jetson_with_dtb_sigheader.bin.encrypt of size 2986048
[   0.1188 ] Added binary blob_pscfw_t234_prod_sigheader.bin.encrypt of size 375168
[   0.1228 ] Added binary blob_mce_flash_o10_cr_prod_sigheader.bin.encrypt of size 190592
[   0.1262 ] Added binary blob_applet_t234_sigheader.bin.encrypt of size 277312
[   0.1298 ] Not supported type: mb2_applet
[   0.1312 ] Added binary blob_mb2_t234_with_mb2_cold_boot_bct_MB2_sigheader.bin.encrypt of size 438768
[   0.1361 ] Added binary blob_xusb_t234_prod_sigheader.bin.encrypt of size 164864
[   0.1396 ] Added binary blob_display-t234-dce_sigheader.bin.encrypt of size 9097216
[   0.1429 ] Added binary blob_nvdec_t234_prod_sigheader.fw.encrypt of size 294912
[   0.1460 ] Added binary blob_bpmp_t234-TE992M-A1_prod_sigheader.bin.encrypt of size 1070208
[   0.1504 ] Added binary blob_fsi-fw-ecc_sigheader.bin.encrypt of size 5746688
[   0.1534 ] Added binary blob_tegra234-bpmp-3701-0008-3737-0000_with_odm_sigheader.dtb.encrypt of size 151360
[   0.1577 ] Added binary blob_camera-rtcpu-sce_sigheader.img.encrypt of size 166304
[   0.1610 ] Added binary blob_camera-rtcpu-t234-rce_sigheader.img.encrypt of size 537952
[   0.1645 ] Added binary blob_adsp-fw_sigheader.bin.encrypt of size 400864
[   0.1675 ] Added binary blob_spe_t234_sigheader.bin.encrypt of size 270336
[   0.1710 ] Added binary blob_tos-optee_t234_sigheader.img.encrypt of size 1127568
[   0.1747 ] Added binary blob_eks_t234_sigheader.img.encrypt of size 9232
[   0.1781 ] Added binary blob_boot.img of size 44480512
[   0.1818 ] Added binary blob_tegra234-orin-agxi-cti-AGX202.dtb of size 336203
[   0.2537 ] tegrarcm_v2 --chip 0x23 0 --pollbl --download bct_mem mem_rcm_sigheader.bct.encrypt --download blob blob.bin
[   0.2555 ] BL: version 1.2.0.0-t234-54845784-562369e5 last_boot_error: 0
[   0.3191 ] Sending bct_mem
[   0.3205 ] Sending blob
[   5.4659 ] RCM-boot started

And then it stops. It does not appear to have flashed the NVMe SSD. Am I missing a step somewhere?

2

I think I found what I was doing wrong to get to where I was above. I have a couple of different Linux_for_Tegra folders as I’m playing with this, and I ran l4t_initrd_flash.sh from an older one. Running it from the correct directory still hits the “no-flash flag enabled” message, but then it keeps going from there instead of stopping.

Now I get to the end and it says

Success
Cleaning up…
Finish generating flash package.
No devices to flash

When I enter the command lsusb I do see the device:

Bus 001 Device 089: ID 0955:7035 NVIDIA Corp.

When I am trying to flash nvme0n1p1, is that looking for something connected to the host machine at the address /dev/nvme0n1p1, or is it looking for something connected to the Orin through the Rogue carrier board at that address?

3

I’ve maybe gotten a little further, but am more puzzled now than before. After reading elsewhere on this forum that even if you’re going to flash to NVMe, you still have to flash to eMMC first, I removed the --external-only flag from my flash command. Flashing finished with a “Success” message, but when the Orin went to boot up, it got hung up and on my serial debug output, I see [ 18.802645] ERROR: nvme0n1p1 not found

Further, in my flash log, it says it successfully flashed to qspi. But I’m not trying to flash to qspi, I’m trying to flash to NVMe. I’ve attached the full log.

flash_1-7.3_0_20240319-121753.log (39.7 KB)

Before doing this, it used to boot up fine from eMMC, now it won’t boot at all.

5

This is not required.
Please just keep flash_l4t_external.xml as is.

This is not true.
eMMC has nothing to do with NVMe.

Combined with what I said above, it is the QSPI bootloader that needs to be flashed, not eMMC.

Formatting APP partition /dev/sdb1 …

I don’t know why you are getting sdb here.
For rel-35, run:

sudo ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1
-c tools/kernel_flash/flash_l4t_external.xml -p “-c bootloader/t186ref/cfg/flash_t234_qspi.xml”
–showlogs --network usb0 <board config> internal

For rel-36, run:

sudo ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1
-c tools/kernel_flash/flash_l4t_external.xml -p “-c bootloader/generic/cfg/flash_t234_qspi.xml”
–showlogs --network usb0 <board config> internal

Give me the flashing log and also this:

6

First, thanks for responding. Second, if modifying the XML file isn’t required, somebody needs to change README_initrd_flash.txt, which explicitly says the following:

Especially note that you will need to change the “num_sectors” field of
the partition config xml file to match your external storage device, as Initrd
flash does not support size discovery. You must change “num_sectors” so that
num_sectors * sector_size is equal to or smaller the size of your external
storage device. And for all types of external device, the device “type” needs to
be “nvme”.

Third, I don’t know what rel -35 or rel -36 means. I’ll try them both and report back with what happened.

7

JetPack 5/rel-35, JetPack 6/rel-36.

You only need to do this when the disk size is smaller than the default value, which is set to be 64GB.
The logic is that flashing scripts will still treat the disk as a 64GB one no matter how large it is, and you indeed only get 64GB upon first boot, but we have a system service that will enlarge the rootfs to make it take all disk space, so it doesn’t really matter.

1 Like
8

I ran the command

sudo ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 -c tools/kernel_flash/flash_l4t_external.xml -p "-c bootloader/t186ref/cfg/flash_t234_qspi.xml" --showlogs --network usb0 cti/orin-agxi/rogue-orin/base internal

Here is the flash log.
flash_1-7.3_0_20240320-112943.log (41.2 KB)
And here is the minicom output.
log202403201142.txt (148.5 KB)

9

Then are you able to boot it via eMMC?

10

It is not booting at all now.

11

Then please flash again with flash.sh and see what you get.

sudo ./flash.sh <board config> internal

Contact the vendor if it does not work this way.

12

After that, it boots from eMMC fine. Now, how do I get it to boot from NVMe instead?

13

I think I got it, but I don’t know if it was just this last thing I tried, or if it was the series of things leading up to it. My final command was

sudo ADDITIONAL_DTB_OVERLAY_OPT="BootOrderNvme.dtbo" ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 -c ./tools/kernel_flash/flash_l4t_external.xml  --showlogs  cti/orin-agxi/rogue-orin/base nvme0n1p1

The only difference between the above command and one that I had tried earlier in this thread is that I’m not using my custom XML file, just the standard flash_l4t_external.xml file. I’m going to see if I can reproduce this from scratch now.

1 Like
14

Adjust the boot order in UEFI menu.

15

sudo ADDITIONAL_DTB_OVERLAY_OPT=“BootOrderNvme.dtbo” ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 -c ./tools/kernel_flash/flash_l4t_external.xml --showlogs cti/orin-agxi/rogue-orin/base nvme0n1p1

This command has been the most successful thing I’ve done, but it seems to work roughly 2/3 of the time. The other 1/3 of the times I run this, I get a “Successfully flash the qspi” message, but then an error flashing the non-qspi memory immediately after. On successful runs, the final output before rebooting reads:

[ 228]: l4t_flash_from_kernel: Successfully flash the qspi
[ 228]: l4t_flash_from_kernel: Flashing success
tar: Read checkpoint 750000
tar: Read checkpoint 760000
tar: Read checkpoint 770000
writing item=16, 9:0:secondary_gpt, 62545444352, 16896, gpt_secondary_9_0.bin, 16896, fixed-<reserved>-0, c9d335548b5a6ebca92b0374fcde8f51e9a71350
[ 324]: l4t_flash_from_kernel: Successfully flash the external device
[ 324]: l4t_flash_from_kernel: Flashing success
[ 325]: l4t_flash_from_kernel: The device size indicated in the partition layout xml is smaller than the actual size. This utility will try to fix the GPT.
17

You should add “--network usb0” to the command. It uses NFS for flashing the NVMe which is more reliable.

18

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.