Failure when using initrd_flash to flash custom JetPack 5.1.4 image with A/B boot on external NVMe

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Xavier
,
1

I’m working with JetPack 5.1.4, BSP R35.6.0 and an AGX Xavier with a 32 GB internal drive The external NVMe hard drive is 4TB. lsblk shows them as mmcblk0 29.1G and nvme0n1 3.7T. fdisk shows the NVMe as Disk /dev/nvme0n1: 3.65 TiB, 4000787030016 bytes, 7814037168 sectors

I was able to flash the default A/B boot settings to the NVMe external drive and was able to boot into both the A and B partitions. Using this comand

sudo ADDITIONAL_DTB_OVERLAY_OPT="BootOrderNvme.dtbo" ROOTFS_AB=1 ROOTFS_RETRY_COUNT_MAX=3 ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 --external-only -c ./tools/kernel_flash/flash_l4t_t194_nvme_rootfs_ab.xml --showlogs  jetson-xavier external

I then updated the rootfs directory with the contents of the custom image I want to flash to each partition. This image is 28 GB. I’ve also added some extra partitions that we need for our product. Details below.

THE ISSUE: Now each time I run the above script, it fails with the following error

***************************************
*                                     *
*  Step 3: Start the flashing process *
*                                     *
***************************************
...
Waiting for target to boot-up...
Waiting for device to expose ssh ......Error: ipv6: address already assigned.
Error: ipv6: address already assigned.
8388608
[ 0]: l4t_flash_from_kernel: Warning: serial number sysfs node doesn't exist.
[ 0]: l4t_flash_from_kernel: Starting to create gpt for external device
Active index file is /Jetson_Linux_R35.6.0_aarch64/Linux_for_Tegra/tools/kernel_flash/images/external/flash.idx
Number of lines is 19
max_index=18
writing item=1, 9:0:primary_gpt, 512, 19968, gpt_primary_9_0.bin, 16896, fixed-<reserved>-0, ccb0cfa196cde3fcff5a0b59614ec8e607156f6d
Writing primary_gpt partition with gpt_primary_9_0.bin
Offset is not aligned to K Bytes, no optimization is applied
dd if=/Jetson_Linux_R35.6.0_aarch64/Linux_for_Tegra/tools/kernel_flash/images/external/gpt_primary_9_0.bin of=/dev/sdd bs=1 skip=0  seek=512 count=16896
16896+0 records in
16896+0 records out
16896 bytes (17 kB, 16 KiB) copied, 0.0166788 s, 1.0 MB/s
Writing primary_gpt partition done
Error: Invalid argument during seek for read on /dev/sdd
[ 0]: l4t_flash_from_kernel: Error: partprobe failed. This indicates that:
 -   the xml indicates the gpt is larger than the device storage
 -   the xml might be invalid
 -   the device might have a problem.
     Please make correction.
Error flashing non-qspi storage

Cleaning up...

Through reading the documentation and trial and error I updated the command to the following, which still gives the same error.

sudo BOARDID=2888 BOARDSKU=0004 FAB=401 BOARDREV=B.0 ADDITIONAL_DTB_OVERLAY_OPT="BootOrderNvme.dtbo" ROOTFS_AB=1 ROOTFS_RETRY_COUNT_MAX=3 ./tools/kernel_flash/l4t_initrd_flash.sh -S 69999996928 --external-device nvme0n1p1 --external-only -c ./tools/kernel_flash/flash_l4t_t194_nvme_rootfs_ab.xml  --showlogs  jetson-xavier external

What I have tried so far

  1. I have updated the flash_l4t_t194_nvme_rootfs_ab.xml with the number of sectors to match the 4TB NVMe: 7814037168.
  2. I added the -S flag with a 70 GB size to cover the image and have extra space in the partition.
  3. Tried the -S flag with 29 GB to barely hold the image
  4. I verified that the partitions were created on the NVMe drive, though the video partition, a new partition I added, was about 1/3 the size it should have been. I tried making it less than 1 TB in case that mattered. Didn’t help.
  5. I verified that the total size of all partitions is less than than what the NVMe max is. I even reduced the partitions to be less than 2 TB total to verify it wasn’t running out of space.
  6. I tried using the original flash_l4t_t194_nvme_rootfs_ab.xml without my added partitions and only the number of sectors updated to match the NVMe size
  7. I have gone through all of the flags and tried all of the ones that seem relevant including the ones listed above and --direct, --flash-only, EXTDEV_ON_TARGET=“/dev/nvme0n1”
  8. The only flag that gave a different error was --network usb0 which gave the error: SSH is not ready

Every combination results in the same error.
Here is my updated XML file in case it helps.
flash_l4t_nvme_rootfs_ab.xml.txt (14.3 KB)

Any help with understanding what’s wrong and getting this to flash properly would be appreciated.

3

hello djh1,

please check $OUT/Linux_for_Tegra/tools/kernel_flash/README_initrd_flash.txt for [Workflow 3: How to flash to an external storage], you should set rootdev as NVMe.

besides, the size units (-S option) in bytes, for example, 1GiB means 1024 * 1024 * 1024 bytes.
did you update the EXT_NUM_SECTORS from xml file? you should update EXT_NUM_SECTORS with a number.
so that 512 * EXT_NUM_SECTORS = 4TB.