Jetson Xavier NX: NVMe Backup + Restore Done But Not Booting From NVMe

[mohit.talwar]

Hi all,

I’m working with a Jetson Xavier NX Devkit and need help with restoring a working NVMe backup to a fresh board.

Backup Process

On my working Xavier NX, I had everything set up:

• JetPack 5.1.3 (flashed via SDK Manager, using NVMe as rootfs)
• DeepStream, OpenCV (built with CUDA), custom libraries, and my full codebase

I followed this thread to create a backup:

Clone one NVMe and flash onto other jetson xavier nx

Hi, we do have an official way to clone NVMe SSDs, and refer to /Linux_for_Tegra/tools/backup-restore/README_backup_restore.txt for detailed instructions. Note that currently you have to manually replace mmcblk0 with nvme0n1 in nvbackup_partitions.sh and nvrestore_partitions.sh for this method to work with NVMe SSDs. We’ll add an option for users to select which storage device to clone in the upcoming L4T release.

• I did not rename mmcblk to nvme in the script, assuming it’s fixed in JetPack 5.1.3.

$ sudo ./tools/backup_restore/l4t_backup_restore.sh -e nvme0n1 -b jetson-xavier-nx-devkit

Backup using nv_backup_partitions.sh completed successfully.

• The .tar.gz file is about 78.1 GB

• The generated partition table also seems fine (attached in my post)

nvpartitionmap.txt (1.8 KB)

Restore Process

I used nv_restore_partitions.sh on a fresh Jetson Xavier NX Devkit:

• It has JetPack 5.1.3 on eMMC
• The NVMe is new and empty

sudo ./tools/backup_restore/l4t_backup_restore.sh -e nvme0n1 -r jetson-xavier-nx-devkit

• Restore completed without any errors

33554432 bytes (34 MB, 32 MiB) copied, 2.52372 s, 13.3 MB/s
nvrestore_partitions.sh Restoring nvme0n1 with image nvme0n1p5_bak.img…
131072+0 records in
131072+0 records out
67108864 bytes (67 MB, 64 MiB) copied, 5.00121 s, 13.4 MB/s
nvrestore_partitions.sh Restoring nvme0n1 with image nvme0n1p6_bak.img…
896+0 records in
896+0 records out
458752 bytes (459 kB, 448 KiB) copied, 0.0376251 s, 12.2 MB/s
nvrestore_partitions.sh Restoring nvme0n1 with image nvme0n1p7_bak.img…
65536+0 records in
65536+0 records out
33554432 bytes (34 MB, 32 MiB) copied, 2.51688 s, 13.3 MB/s
nvrestore_partitions.sh Restoring nvme0n1 with image nvme0n1p8_bak.img…
163840+0 records in
163840+0 records out
83886080 bytes (84 MB, 80 MiB) copied, 6.26612 s, 13.4 MB/s
nvrestore_partitions.sh Restoring nvme0n1 with image nvme0n1p9_bak.img…
1024+0 records in
1024+0 records out
524288 bytes (524 kB, 512 KiB) copied, 0.0419881 s, 12.5 MB/s
nvrestore_partitions.sh Restoring nvme0n1 with image nvme0n1p10_bak.img…
204800+0 records in
204800+0 records out
104857600 bytes (105 MB, 100 MiB) copied, 7.81574 s, 13.4 MB/s
nvrestore_partitions.sh Restoring nvme0n1 with image nvme0n1p11_bak.img…
131072+0 records in
131072+0 records out
67108864 bytes (67 MB, 64 MiB) copied, 4.94964 s, 13.6 MB/s
nvrestore_partitions.sh Restoring nvme0n1 with image nvme0n1p12_bak.img…
163840+0 records in
163840+0 records out
83886080 bytes (84 MB, 80 MiB) copied, 6.19052 s, 13.6 MB/s
nvrestore_partitions.sh Restoring nvme0n1 with image nvme0n1p13_bak.img…
1024+0 records in
1024+0 records out
524288 bytes (524 kB, 512 KiB) copied, 0.0419075 s, 12.5 MB/s
nvrestore_partitions.sh Restoring nvme0n1 with image nvme0n1p14_bak.img…
131072+0 records in
131072+0 records out
67108864 bytes (67 MB, 64 MiB) copied, 4.9238 s, 13.6 MB/s
~
nvrestore_partitions.sh Successful restore of partitions on target board.
Operation finishes. You can manually reset the device

The Problem

After restoring, the Jetson does not boot from NVMe.

i get this :

What i think is that while restoring some nvme configuration is mismatched since different nvme and thats why it is not able to boot.

This is one of the issue i have faced , apart from this i have tried various other combinations but nothing is working . I am going in the wrong direction.

Can you give me a structured approach to fix this issue

Thanks :)

[DavidDDD]

Hi,

Some questions to confirm:

• Could you confirm whether both NVMe have the same storage size?
• Please provide the serial console log for our review.

Jetson Nano & NX Style - Serial Debug Console - JetsonHacks

Thanks

[mohit.talwar]

• No, both NVMe SSDs are not the same size. The Jetson Xavier NX I backed up had a 1TB NVMe SSD, while the ones I’m restoring to are smaller — one is around 500GB and the other around 230GB (I’ve tried both).
• What log exactly do you need?
  I don’t have a debug UART cable, but if I can share logs via recovery mode and micro-USB, let me know and I’ll do that. I am currently reflashing the backup image after formating and making partition table again via booting into emmc and formating the nvme.

[DavidDDD]

@mohit.talwar

No, both NVMe SSDs are not the same size. The Jetson Xavier NX I backed up had a 1TB NVMe SSD, while the ones I’m restoring to are smaller — one is around 500GB and the other around 230GB (I’ve tried both).

It might be the error come from.
Please do below steps to do the NVMe Backup + Restore

1. flash your Xavier NX with argument -S 500GiB to constraint the rootfs size
2. backup the NVMe image
3. And restore

Please make sure you target restore one size is larger or equal to the image size you backup.

Since you are using sdk manager to flash your Xavier, please refer to related topics Flashing Xavier NX with NVME ssd

sudo ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 -c tools/kernel_flash/flash_l4t_external.xml -p -S 500GiB "-c bootloader/t186ref/cfg/flash_l4t_t194_qspi_p3668.xml" –showlogs --network usb0 jetson-xavier-nx-devkit-emmc internal

Thanks