Jetson AGX Thor (R39.2.0) Backup & Massflash Workflow - Multiple Issues

I am trying to create a backup/clone of a configured Jetson AGX Thor (R39.2.0) originally installed via the official USB stick method and flash it onto another Thor device, but the official Workflow (l4t_backup_restore.sh + l4t_initrd_flash.sh) produces a non-bootable result.

Setup:

  • Host PC: Ubuntu 24.04
  • Jetson AGX Thor Developer Kit (p4071-0000 + p3834-0008, L4T R39.2.0 / JetPack 7.2)
  • Initial installation: USB stick method as described in the Quick Start Guide, including QSPI firmware update on first boot
  • After setup: Ubuntu 24.04 with ROS 2 Jazzy, RT kernel, and various drivers installed
  • Goal: Clone this configured image for deployment on multiple Thors (same approach worked on Orin/R36.x without issues)

Procedure followed: NVIDIA Backup & Restore Guide (R39.2) + Workflow 3 from tools/backup_restore/README_backup_restore.txt


Step 1: Prerequisites

sudo systemctl stop udisks2.service
sudo service nfs-kernel-server start
sudo ./tools/l4t_flash_prerequisites.sh
sudo ./apply_binaries.sh

Step 2: Backup

sudo ./tools/backup_restore/l4t_backup_restore.sh -b -c jetson-agx-thor-devkit

Backup completes successfully. Images stored in tools/backup_restore/images/.


Step 3: Generate massflash package - Multiple Issues

As per Workflow 3 README:

sudo ./tools/kernel_flash/l4t_initrd_flash.sh \
    --use-backup-image --no-flash --network usb0 \
    --massflash 5 jetson-agx-thor-devkit internal

Issue 3a: EngSample=True in generated TargetInfo.txt

The generated unified_flash/out/bsp_images/flash_workspace/TargetInfo.txt contains EngSample=True, causing flashing to fail on production-silicon devices:

Aborting, command option -M not allowed with PROD board

Root cause: FUSELEVEL defaults to fuselevel_nofuse for the developer kit, which causes create_l4t_bsp_images.py --internal to be invoked, setting EngSample=True.

Workaround: Set FUSELEVEL=fuselevel_production explicitly:

sudo FUSELEVEL=fuselevel_production \
    ./tools/kernel_flash/l4t_initrd_flash.sh \
    --use-backup-image --no-flash --network usb0 \
    --massflash 5 jetson-agx-thor-devkit internal

Issue 3b: s_Skuname empty in jetson-t264.json

After --no-flash, unified_flash/out/bsp_images/tools/flashtools/board_configs/jetson-t264.json contains:

{"jetson-t264": {"s_RamCode": "12", "s_Skuname": ""}}

This causes:

Mismatched Sku TA1090SA

Workaround:

echo '{"jetson-t264": {"s_RamCode": "12", "s_Skuname": "TA1090SA"}}' | sudo tee \
    unified_flash/out/bsp_images/tools/flashtools/board_configs/jetson-t264.json

Issue 3c: flash-images directory not generated

When using --use-backup-image, reuse_package=1 is set internally, which skips the flash-images generation block in l4t_initrd_flash_internal.sh (line ~1079). Without flash-images, flash_bsp_images.py fails with:

FileNotFoundError: flash_workspace/flash-images

Additionally, when --use-backup-image is NOT used (to allow flash-images generation), generate_flash_package() overwrites tools/kernel_flash/images/external/ with fresh BSP images, destroying the backup partition data.

Workaround we used:

  1. Run without --use-backup-image to generate flash-images but this produces a fresh Ubuntu install, not the cloned image
  2. Manually re-convert backup images using convert_backup_image_to_initrd_flash and then call create_l4t_bsp_images.py directly

Question: What is the correct procedure to generate flash-images from a backup while preserving the backup partition data?

Step 4: Flash the package onto a target device - Python bug + non-bootable result

sudo ./tools/kernel_flash/l4t_initrd_flash.sh \
    --flash-only --massflash 5 --network usb0 \
    jetson-agx-thor-devkit internal

Issue 4a: UnboundLocalError: abs_path in bootburn_adb.py

While flashing the backup’s APP partition (which has Start=0), resizeFilesystem() crashes:

File "bootburn_t264_py/bootburn_adb.py", line 904, in resizeFilesystem
    adbOutput = self.AdbShell(abs_path + "e2fsck -yfFt " + LocalPartition)
UnboundLocalError: cannot access local variable 'abs_path' where it is not associated with a value

Root cause: In resizeFilesystem(), abs_path is only assigned inside the if (int(partitionInfo.Start) != 0): block, but is used unconditionally afterwards for the e2fsck / resize2fs calls. For a partition with Start=0, abs_path is never defined.

Workaround: Add abs_path = "" before the if block (must be patched in both unified_flash/tools/flashtools/bootburn_t264_py/bootburn_adb.py and the copy under unified_flash/out/bsp_images/tools/...).

After this patch, flashing completes:

[flash_bsp_jetson-t264]: Flashing finished Successfully!!

Issue 4b: Device does not boot after successful flash - drops to UEFI Shell

Despite the successful flash, the device boots into the UEFI Shell instead of Ubuntu. Investigating from the shell:

  • map -r shows only FS0:, FS1, BLK0 - the NVMe partitions are largely not mapped as filesystems

  • bcfg boot dump shows no Ubuntu / Linux boot entry - only generic entries:

    UEFI HTTPv6 / HTTPv4 / PXEv6 / PXEv4
    UEFI WD PC SN5000S       (the NVMe SSD)
    UEFI Shell
    
  • Selecting UEFI WD PC SN5000S from the Boot Manager briefly refreshes and returns to the menu (does not boot)

  • Browsing the mapped filesystems shows no EFI\ubuntu\ or EFI\BOOT\BOOTAA64.EFI - the EFI System Partition appears to be missing or not correctly populated from the backup

This suggests the backup → massflash conversion does not correctly restore/populate the ESP for Thor.


Step 5: Direct restore to a single device - fails at RCM boot (USB timeout)

As an alternative to massflash, we tried restoring the backup directly to a single device (Workflow 2):

sudo ./tools/backup_restore/l4t_backup_restore.sh -r jetson-agx-thor-devkit

Issue 5a: cvm.bin does not exist (EEPROM read fails)

The EEPROM read step fails with a USB timeout, and because the tool renames cvm.bin to cvm.bin.sav before the (failing) read, the restore aborts:

[   3.0448 ] tegrarcm_v2 --chip 0x26 0 --pollbl --download applet applet_t264_sigheader.bin.encrypt
[   3.0450 ] ERROR: might be timeout in USB read
...
cvm.bin does not exist

Workaround: Restore the saved EEPROM data and skip the EEPROM check:

sudo cp bootloader/cvm.bin.sav bootloader/cvm.bin
sudo cp bootloader/custinfo_out.bin.sav bootloader/custinfo_out.bin
sudo SKIP_EEPROM_CHECK=1 BOARDID=3834 BOARDSKU=0008 FAB=400 BOARDREV=H.1 \
    ./tools/backup_restore/l4t_backup_restore.sh -r jetson-agx-thor-devkit

Issue 5b: RCM applet download times out

With the EEPROM check skipped, the flash package generates successfully, but the actual RCM boot of the flashing initrd fails at the applet download:

tegrarcm_v2 --instance /dev/bus/usb/003/027 --chip 0x26 --pollbl --download applet applet/applet_blob.bin
ERROR: might be timeout in USB read
Return code: 8
TEGRARCM -- Unable to execute command
OSError: 43

Note: The same --pollbl --download applet USB timeout appears intermittently in the --flash-only path too, but that workflow apparently tolerates/retries it, whereas the rcm-boot path (used by restore) fails hard.


Open questions:

  1. What is the correct, supported procedure to create a bootable clone/massflash image of a configured Jetson AGX Thor from a l4t_backup_restore.sh -b -c backup? Specifically, how is the EFI System Partition supposed to be preserved and restored?
  2. Are the EngSample/FUSELEVEL, empty s_Skuname, and abs_path issues known bugs for R39.2.0, or are we using the wrong invocation?
  3. Is --use-backup-image --no-flash expected to work for the Thor / UNIFIED_FLASH flashing architecture? If so, what is the complete correct command sequence?
  4. What causes the tegrarcm_v2 --pollbl --download applet USB timeout specifically on the rcm-boot path, and how can it be avoided (cable/port aside)?

Hi,

Quick question to confirm:

  • Is your Thor using a developer kit or a custom board?

Thanks

Hi, it is a developer kit.

Update: Further investigation with SDK Manager base installation

To isolate whether the issue was specific to the USB stick installation method, we repeated the full workflow using an SDK Manager (JetPack 7.2) base installation. All issues were reproducible identically, confirming they are independent of the initial installation method.

Additional finding - Step 4: Backup images are Zstandard-compressed, not raw

All _bak.img files generated by l4t_backup_restore.sh -b -c are Zstandard-compressed:

$ file tools/backup_restore/images/nvme0n1p4_bak.img
nvme0n1p4_bak.img: Zstandard compressed data (v0.8+), Dictionary ID: None
Size: 65468 bytes compressed → 536870912 bytes (512MB) decompressed

The FileToFlash.txt in the generated flash-images directory references these compressed files and stores the MD5 of the compressed file, not the decompressed partition data. Investigation suggests nvdd on the Thor’s flashing initrd may write the compressed bytes directly to the NVMe partition instead of decompressing first, resulting in all NVMe partitions containing Zstandard data rather than valid filesystems.

Test: After manually decompressing all _bak.img files and updating the MD5 checksums in FileToFlash.txt, re-flashing completes successfully but the device still boots to UEFI Shell with no EFI bootloader found.

Additional finding: ESP not visible after flash

After flash (both with compressed and decompressed images), map -r in the UEFI Shell shows only:

  • FS0: QSPI-based UEFI firmware partitions
  • FS1: one unidentified NVMe partition (contains UUID-named files)
  • BLK0: raw NVMe block device

The EFI System Partition (nvme0n1p4, 512MB FAT32) is never visible as a filesystem in UEFI. Browsing all mapped filesystems confirms EFI\BOOT\BOOTAA64.EFI is not accessible.

The backup nvme0n1p4_bak.img when decompressed IS a valid 512MB FAT32 containing EFI/BOOT/BOOTAA64.efi:

$ file nvme0n1p4_bak.img # after decompression
DOS/MBR boot sector, OEM-ID “mkfs.fat”, FAT (32 bit), …
$ # Mounted: contains /EFI/BOOT/BOOTAA64.efi ✓

Question: Is the nvme0n1p4 EFI System Partition type GUID (C12A7328-F81F-11D2-BA4B-00A0C93EC93B) preserved in the GPT backup? And does the flash workflow restore the GPT partition type correctly? We suspect the partition type GUID for the ESP may not be preserved/restored, causing UEFI to not recognize it as a bootable ESP.

Hi,

Please apply the patch provided under the Linux_for_Tegra

Linux_for_Tegra$ patch -p1 < r39.2_Testing-massflash-backup.patch

r39.2_Testing-massflash-backup.patch (44.4 KB)

Thanks