Using the initrd tool to flash the image extracted from backup_restore

Hello everyone,

I am trying the method provided in the link below on JetPack 6: Backup restore, workflow #3 massflash jetson orin nano problem

After modifying nvbackup_partitions.sh, nvrestore_partitions.sh, and l4t_backup_restore.func, I extracted the image using the following command:

$ sudo ./tools/backup_restore/l4t_backup_restore.sh -e nvme0n1 -b -c jetson-agx-orin-devkit

Then I used initrd to flash the image:

$ sudo BOARDID=3701 FAB=500 BOARDSKU=0005 ./tools/kernel_flash/l4t_initrd_flash.sh --use-backup-image --no-flash --external-device nvme0n1p1 -c tools/kernel_flash/flash_l4t_external.xml -p "-c bootloader/generic/cfg/flash_t234_qspi.xml --no-systemimg" --showlogs --network eth0:192.168.0.61/24:192.168.0.80 jetson-agx-orin-devkit external
$ sudo ./tools/kernel_flash/l4t_initrd_flash.sh --flash-only --external-device nvme0n1p1 -c tools/kernel_flash/flash_l4t_external.xml -p "-c bootloader/generic/cfg/flash_t234_qspi.xml --no-systemimg" --showlogs --network eth0:192.168.0.61/24:192.168.0.80 jetson-agx-orin-devkit nvme0n1p1

An error message occurred during the process:

Linux_for_Tegra/tools/kernel_flash/images/internal/flash.cfg': No such file or directory

The detailed flashing process is recorded in the attached log.txt file.

log.txt (92.6 KB)

I understand that this flashing method is expected to be supported in JetPack 6.1, but my superior hopes that I can complete this flashing process as soon as possible. Therefore, I would like to ask if anyone has any suggestions to resolve the issue mentioned above.

Thank you.

Hi,

How does your Linux_for_Tegra/tools/kernel_flash/images/internal/ and /external/ look like?
The patch is only verified on JetPack 5, and seems to have some compatibility issues on JetPack 6.
I didn’t fully test it.

Hi DaveYYY,

Thank you for quick answer, this is the content of the external folder

(base) ceslab-onyx@ceslab-onyx-server:/mnt/nvme1n1p2/JS1100/Linux_for_Tegra/tools/kernel_flash/images/external$ ls -lah
total 112G
drwxr-xr-x 2 root root 4.0K  六  20 14:07 .
drwxr-xr-x 4 root root 4.0K  六  20 14:07 ..
-rw-r--r-- 1 root root 2.1K  六  20 14:07 flash.idx
-rw-r--r-- 1 root root  17K  六  20 14:03 gptbackup.img
-rw-r--r-- 1 root root  20K  六  20 14:03 gptmbr.img
-rw-r--r-- 1 root root  64M  六  20 14:03 nvme0n1p10_bak.img
-rw-r--r-- 1 root root  80M  六  20 14:03 nvme0n1p11_bak.img
-rw-r--r-- 1 root root 1.0M  六  20 14:03 nvme0n1p12_bak.img
-rw-r--r-- 1 root root  64M  六  20 14:03 nvme0n1p13_bak.img
-rw-r--r-- 1 root root 400M  六  20 14:03 nvme0n1p14_bak.img
-rw-r--r-- 1 root root 480M  六  20 14:03 nvme0n1p15_bak.img
-rw-r--r-- 1 root root 111G  六  20 14:03 nvme0n1p1_bak.img
-rw-r--r-- 1 root root 128M  六  20 14:03 nvme0n1p2_bak.img
-rw-r--r-- 1 root root 1.0M  六  20 14:03 nvme0n1p3_bak.img
-rw-r--r-- 1 root root  32M  六  20 14:03 nvme0n1p4_bak.img
-rw-r--r-- 1 root root 128M  六  20 14:03 nvme0n1p5_bak.img
-rw-r--r-- 1 root root 1.0M  六  20 14:03 nvme0n1p6_bak.img
-rw-r--r-- 1 root root  32M  六  20 14:03 nvme0n1p7_bak.img
-rw-r--r-- 1 root root  80M  六  20 14:03 nvme0n1p8_bak.img
-rw-r--r-- 1 root root 1.0M  六  20 14:03 nvme0n1p9_bak.img

this is the content of the internal folder

(base) ceslab-onyx@ceslab-onyx-server:/mnt/nvme1n1p2/JS1100/Linux_for_Tegra/tools/kernel_flash/images/internal$ ls -lah
total 128M
drwxr-xr-x 2 root root 4.0K  六  20 14:07 .
drwxr-xr-x 4 root root 4.0K  六  20 14:07 ..
-rw-r--r-- 1 root root  32M  六  20 14:03 boot0.img
-rw-r--r-- 1 root root  32M  六  20 14:03 boot1.img
-rw-r--r-- 1 root root  106  六  20 14:07 flash.idx
-rw-r--r-- 1 root root 1.9K  六  20 14:03 nvpartitionmap.txt
-rw-r--r-- 1 root root  64M  六  20 14:03 QSPI0.img

Can you also show file contents of flash.idx in both folders, and also nvpartitionmap.txt?

Here is what you requested. I hope it helps
BTW, since the .idx file cannot be uploaded, I have temporarily changed it to a .txt file.

flash_internal.txt (106 Bytes)
nvpartitionmap_internal.txt (1.9 KB)
flash_external.txt (2.0 KB)

OK, so all these files look expected.
I’m not sure why flash.cfg is not generated, but you can manually make one.

For /internal/flash.cfg, write:

external_device=nvme0n1p1

For /external/flash.cfg, write:

APP_ext=nvme0n1p1_bak.img
external_device=nvme0n1p1
1 Like

Hi DaveYYY,

I’m very sorry, the disappearance of the flash.cfg file was my mistake.

After checking, I found that in l4t_backup_restore.func on JetPack 6, a new command was added:

touch "${destination}/internal/flash.cfg"

I didn’t notice this and overwrote this line of code when making modifications.

After fixing the error, I re-flashed the device. However, during the flashing process, I encountered the following errors:

Warning: Unable to open /dev read-write (Is a directory).  /dev has been opened read-only.
Error: device is too small for GPT

I used the same device for both extracting the image and flashing the image.
The attachment contains the error messages from the command window

cmd_message.txt (7.2 KB)

Please review them. I look forward to your reply.

Are you sure this is the right IP address when the device boots into initrd?

–network eth0:192.168.0.61/24:192.168.0.80

Can you switch to --network usb0 and try again?

Please also show the serial console log.

Hi DaveYYY,

I can confirm that the IP address is correct. After the device enters RCM, I am able to connect to it via SSH.

Additionally, I followed your advice and modified the network parameters, setting the parameter to usb0. However, I encountered the same issue:

Error: device is too small for GPT.

Attached is the serial console log for your reference. I look forward to your response.
serial console log.txt (39.4 KB)

Best regards,

Evan

[ 17.002704] usb usb2-port2: Cannot enable. Maybe the USB cable is bad?

Can you get another USB cable and try again?

Hi DaveYYY,

I have clarified the issue with the USB. The problem was caused by a program modification I made previously. To avoid similar errors, I am now using a clean BSP and have only modified the Backup_restore code. After repeating the above steps, I still encountered the same error during the flashing process:

$ sudo ./tools/kernel_flash/l4t_initrd_flash.sh --flash-only --external-device nvme0n1p1 -c tools/kernel_flash/flash_l4t_external.xml -p "-c bootloader/generic/cfg/flash_t234_qspi.xml --no-systemimg" --showlogs --network usb0 jetson-agx-orin-devkit external
Warning: Unable to open /dev read-write (Is a directory).  /dev has been opened read-only.
Error: device is too small for GPT
Flash failure

However, in this experiment, the serial console log records are different. Here are the last three lines:

[    9.368386] IPv6: ADDRCONF(NETDEV_CHANGE): usb0: link becomes ready
[    9.371398] hid-generic 0003:046D:C345.0002: input,hidraw1: USB HID v1.11 Keyboard [Logitech Logi TKL Mechanical Keyboard] on usb-3610000.usb-4.1/input1
[    9.374912] hid-generic 0003:046D:C345.0003: hidraw2: USB HID v1.11 Device [Logitech Logi TKL Mechanical Keyboard] on usb-3610000.usb-4.1/input2

The complete serial console log is attached, and I hope it helps.

serial console log_2.txt (38.0 KB)

Looking forward to your reply.

Evan

Hi,

Sorry for the late reply.
Can you please first confirm whether it works if you backup the eMMC with modifying the script?

Hi DaveYYY,

Thank you for your continued responses with possible solutions.

I haven’t tested whether the modified script can back up eMMC.

The reason is that after changing eMMC to NVMe in nvbackup_partitions.sh and nvrestore_partitions.sh, I’m not sure if it can properly flash eMMC.

However, I found the source of the following error message, which is from Linux_for_Tegra/tools/kernel_flash/images/l4t_flash_from_kernel.sh:

Warning: Unable to open /dev read-write (Is a directory).  /dev has been opened read-only.
Error: device is too small for GPT
Flash failure

The command that generates the error is:

flock -w 60 /var/lock/nvidiainitrdflash parted -s "${disk}" mklabel gpt

The value of ${disk} is /dev/

The code that defines disk is:

disk="/dev/$(get_disk_name "${external_device}")"

I currently believe the error is in the definition of disk, which cannot create a GPT on the expected device (/dev/nvme0n1).

Since I’m not sure how to modify it to maintain the consistency and maintainability of the entire script, I hope you can provide some suggestions.

Looking forward to your reply.

Evan

OK, this makes sense.

Are these two variables correctly specified in flash.cfg?
I feel like this is because external_device is not defined, causing the function get_disk_name() to fail.

Hi DaveYYY,

Thank you for your prompt response.

Firstly, I apologize for not checking the contents of flash.cfg.

After fixing the error that the command to generate flash.cfg was accidentally deleted:

touch "${destination}/internal/flash.cfg"

the flash.cfg file was generated in the internal folder.

However, no flash.cfg was generated in the external folder.

The flash.cfg file in the internal folder does not contain any records.

After creating two flash.cfg files with the content you provided:

For /internal/flash.cfg, write:

external_device=nvme0n1p1

For /external/flash.cfg, write:

APP_ext=nvme0n1p1_bak.img
external_device=nvme0n1p1

I added the flash.cfg files and attempted to flash again.

When executing the initrd flash command, the process got stuck:

sudo ./tools/kernel_flash/l4t_initrd_flash.sh --flash-only --external-device nvme0n1p1 -c tools/kernel_flash/flash_l4t_external.xml -p "-c bootloader/generic/cfg/flash_t234_qspi.xml --no-systemimg" --showlogs --network usb0 jetson-agx-orin-devkit external
118472478720 bytes from /mnt/external/nvme0n1p1_bak.img to /dev/nvme0n1: 1KB block=115695780 remainder=0
dd if=/mnt/external/nvme0n1p1_bak.img of=/dev/nvme0n1 bs=1K skip=0  seek=1525024 count=115695780
Erased 67108864 bytes from address 0x00000000 in flash
Flash index file is /mnt/internal/flash.idx
Number of lines is 1
max_index=0
Writing QSPI0.img (parittion: qspi0) into /dev/mtd0
Sha1 checksum matched for /mnt/internal/QSPI0.img
Writing /mnt/internal/QSPI0.img (67108864 bytes) into  /dev/mtd0:0
 Copied 67108864 bytes from /mnt/internal/QSPI0.img to address 0x00000000 in flash
[ 321]: l4t_flash_from_kernel: Successfully flash the qspi

cmd message 2.txt (9.9 KB)

Evan

Did serial console give any clue? Or did it get any progress if you wait longer?
Or you can add some debug prints in Linux_for_Tegra/tools/kernel_flash/images/l4t_flash_from_kernel.sh to see where the process gets stuck.
Pay attention to the flash_partition() function.

Hi DaveYYY,

Sorry for my haste in uploading the results.

As you mentioned, after waiting a longer period, the burning process was successful.

I really appreciate the help you provided.

However, there is one thing that bothers me during the flashing image, which is the flashing time.

Previously, when using initrd to flsah the image to the device, it took about 10 to 20 minutes.

dd if=<rootfs partition> of=/mnt/system.img.raw
./bootloader/mksparse --fillpattern=0 system.img.raw system.img

But now, using backup_restore to extract the image and then flash it with initrd, it takes at least 50 minutes and at most 2 hours and 20 minutes.

The longest part of the backup process is backing up nvme0n1p1, which takes up more than 80% of the flashing time.

118472478720 bytes (118 GB, 110 GiB) copied, 8332.33 s, 14.2 MB/s
Writing nvme0n1p1 partition done

I guess the previous image might have been compressed with mksparse after extraction.

Is there a way to reduce the size of the image extracted by backup_restore?

Looking forward to your reply.

Evan

Oh sorry, it’s because of a known bug in the backup script (Linux_for_Tegra/tools/backup_restore/nvbackup_partitions.sh) that we have fixed.

isext4() {
	if [ "$#" -ne 1 ]; then
		print_message "isext4 function need 1 parameter that is the name of the storage device"
		return 1;
	fi
	local result
	- result="$( blkid "/dev/${1}" | awk '{ print $3 }' | sed -n 's|TYPE="\(.*\)"|\1|p' )"
	+ result="$( blkid -o value -s TYPE "/dev/${1}" )"
	if [ "${result}" = "ext4" ]; then
		echo "true"
	else
		echo "false"
	fi
}

Please apply this patch so rootfs is correctly backed up using tar instead of dd, which is why backup takes so long with the unmodified script.

Hi DaveYYY:
We followed the steps: backup eMMC → convert to initrd images → flash
All the operations are performed on the same device (AGX 32GB Orin).

This is what we did.

  1. Modify the script, about the isext4() function.

  2. Use the backup restore to backup eMMC, and use -c flag to convert it into initrd images. We did a little changes to avoid some USB timeout issues.

sudo ./tools/backup_restore/l4t_backup_restore.sh --orin-ip $ORIN_IP --host-ip $HOST_IP -c -e mmcblk0 -b jetson-agx-orin-devkit
  1. Use initrd_flash script to flash the device:
$ sudo ./tools/kernel_flash/l4t_initrd_flash.sh --use-backup-image --no-flash --network eth0:192.168.17.52/24:192.168.17.51 --massflash 1 jetson-agx-orin-devkit internal
$ sudo ./tools/kernel_flash/l4t_initrd_flash.sh --flash-only –massflash 1 --network eth0:192.168.17.52/24:192.168.17.51

Note 1: Not sure why the speed of writing the eMMC is only 5MB/s. Will this a overheating issue?
Note 2: During packing MFI package, maybe you can try pigz instead of tar to make the compression process faster.

  1. After flashing and booting, the device is not able to boot. Serial log looks normal, but on screen:

As you can see, the root cause should be the mount error:
Wrong fs type, bad option, bad superblock on /dev/mmcblk0p1, missing codepage or helper program, or other error.

Are there any other clues to trace?

Please help.

Many thanks.

你確定restore回去的時候是成功的嗎?

我們的script應該沒有這樣的option 這是你自己加的?
麻煩先試一下備份完之後單純

sudo ./tools/backup_restore/l4t_backup_restore.sh -e mmcblk0 -r jetson-agx-orin-devkit

還有可以的話麻煩用USB cable 不要用Ethernet 我們很久沒有驗過Ethernet cable flashing了