backup and restore to multiple TX2 in parallel

Hello everyone,

I struggling to restore my backup image to multiple TX2 at the same time (parallelization of flash).

Before beginning my explanation I want to add that I made the whole process with the normal setup (without -r option and own backup described below) and everything worked well. I could flash multiple Jetson at the same time with the default system.img

So let me explain what I tried and what I want to achieve. I have 20 TX2 and I use JetPack 3.3. I flash the first TX2 and install all my environnement (it takes a lot of time). So I make a backup using:

sudo ./flash.sh -r -k APP -G backup.img jetson-tx2 mmcblk0p1

then I want to clone my backup.img to multiple TX2 at the same time. For that I use the following process:

  1. replace the system.img in bootloader (~3.5GB) with my backup.img (~30GB) and rename it system.img (plus deleting the system.img.raw)

  2. then I execute the following commande

sudo ./flash.sh -r --no-flash jetson-tx2 mmcblk0p1

(I added the -r option to keep my system.img and avoid overriding it according to this issue (output of commande at the end).
https://devtalk.nvidia.com/default/topic/1038136/jetson-tx2/about-flashing-multiple-tegra-devices-/2

  1. then I lunch the script to flash on multiple device with
sudo ./multi_tegra_flash.sh

accordingly to this documentation.
https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-282/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fflash_multi.html%23wwpID0EFHA.

the output is the following:

Start flashing; PID=30605
Start flashing device: 1-4, PID: 30653
Start flashing device: 3-2, PID: 30654
Ongoing processes: 30653 30654
Ongoing processes: 30653 30654
Ongoing processes:
Flash complete (WITH FAILURES)

Sincerely
Cortexia Team

BELOW OUTPUT OF

sudo ./flash.sh -r --no-flash jetson-tx2 mmcblk0p1

(as mentioned above)

###############################################################################
# L4T BSP Information:
# R28 (release), REVISION: 2.1, GCID: 11272647, BOARD: t186ref, EABI: aarch64, 
# DATE: Thu May 17 07:29:06 UTC 2018
###############################################################################
./tegraflash.py --chip 0x18 --applet "/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/mb1_recovery_prod.bin" --cmd "dump eeprom boardinfo cvm.bin" --skipuid 
Welcome to Tegra Flash
version 1.0.0
Type ? or help for help and q or quit to exit
Use ! to execute system commands
 
[   0.0017 ] Generating RCM messages
[   0.0023 ] tegrarcm_v2 --listrcm rcm_list.xml --chip 0x18 --download rcm /home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/mb1_recovery_prod.bin 0 0
[   0.0028 ] RCM 0 is saved as rcm_0.rcm
[   0.0031 ] RCM 1 is saved as rcm_1.rcm
[   0.0031 ] List of rcm files are saved in rcm_list.xml
[   0.0031 ] 
[   0.0031 ] Signing RCM messages
[   0.0037 ] tegrasign_v2 --key None --list rcm_list.xml --pubkeyhash pub_key.key
[   0.0043 ] Assuming zero filled SBK key
[   0.0067 ] 
[   0.0067 ] Copying signature to RCM mesages
[   0.0072 ] tegrarcm_v2 --chip 0x18 --updatesig rcm_list_signed.xml
[   0.0080 ] 
[   0.0080 ] Boot Rom communication
[   0.0085 ] tegrarcm_v2 --chip 0x18 --rcm rcm_list_signed.xml --skipuid
[   0.0090 ] RCM version 0X180001
[   0.0098 ] Boot Rom communication completed
[   1.0166 ] 
[   1.0193 ] tegrarcm_v2 --isapplet
[   1.0217 ] Applet version 01.00.0000
[   1.0388 ] 
[   1.0396 ] Retrieving EEPROM data
[   1.0396 ] tegrarcm_v2 --oem platformdetails eeprom cvm /home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/cvm.bin
[   1.0403 ] Applet version 01.00.0000
[   1.0757 ] Saved platform info in /home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/cvm.bin
[   1.1530 ] 
Board ID(3310) version(D01) 
copying bctfile(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/t186ref/BCT/P3310_A00_8GB_Samsung_8GB_lpddr4_204Mhz_A02_l4t.cfg)... done.
copying misc_config(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/t186ref/BCT/tegra186-mb1-bct-misc-si-l4t.cfg)... done.
copying pinmux_config(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/t186ref/BCT/tegra186-mb1-bct-pinmux-quill-p3310-1000-c03.cfg)... done.
copying pmic_config(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/t186ref/BCT/tegra186-mb1-bct-pmic-quill-p3310-1000-c04.cfg)... done.
copying pmc_config(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/t186ref/BCT/tegra186-mb1-bct-pad-quill-p3310-1000-c03.cfg)... done.
copying prod_config(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/t186ref/BCT/tegra186-mb1-bct-prod-quill-p3310-1000-c03.cfg)... done.
copying scr_config(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/t186ref/BCT/minimal_scr.cfg)... done.
copying scr_cold_boot_config(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/t186ref/BCT/mobile_scr.cfg)... done.
copying bootrom_config(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/t186ref/BCT/tegra186-mb1-bct-bootrom-quill-p3310-1000-c03.cfg)... done.
copying dev_params(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/t186ref/BCT/emmc.cfg)... done.
Existing bootloader(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/nvtboot_cpu.bin) reused.
	populating kernel to rootfs... done.
	populating initrd to rootfs... done.
	populating extlinux.conf.emmc to rootfs... done.
	populating /home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/kernel/dtb/tegra186-quill-p3310-1000-c03-00-base.dtb to rootfs... done.
done.
Making Boot image... done.
Existing sosfile(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/mb1_recovery_prod.bin) reused.
copying tegraboot(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/t186ref/nvtboot.bin)... done.
Existing mb2blfile(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/nvtboot_recovery.bin) reused.
Existing mtspreboot(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/preboot_d15_prod_cr.bin) reused.
Existing mts(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/mce_mts_d15_prod_cr.bin) reused.
Existing mb1file(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/mb1_prod.bin) reused.
Existing bpffile(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/bpmp.bin) reused.
copying bpfdtbfile(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/t186ref/tegra186-a02-bpmp-quill-p3310-1000-c04-00-te770d-ucm2.dtb)... done.
Existing scefile(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/camera-rtcpu-sce.bin) reused.
Existing spefile(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/spe.bin) reused.
copying wb0boot(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/t186ref/warmboot.bin)... done.
Existing tosfile(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/tos.img) reused.
Existing eksfile(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/eks.img) reused.
copying dtbfile(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/kernel/dtb/tegra186-quill-p3310-1000-c03-00-base.dtb)... done.
Reusing existing system.img... 
done.
Existing tbcfile(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/cboot.bin) reused.
copying tbcdtbfile(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/kernel/dtb/tegra186-quill-p3310-1000-c03-00-base.dtb)... done.
copying cfgfile(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/t186ref/cfg/flash_l4t_t186.xml) to flash.xml... done.
Existing flasher(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/nvtboot_recovery_cpu.bin) reused.
Existing flashapp(/home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/tegraflash.py) reused.
./tegraflash.py --bl nvtboot_recovery_cpu.bin --sdram_config P3310_A00_8GB_Samsung_8GB_lpddr4_204Mhz_A02_l4t.cfg --odmdata 0x1090000 --applet mb1_recovery_prod.bin --cmd "flash; reboot"  --cfg flash.xml --chip 0x18 --misc_config tegra186-mb1-bct-misc-si-l4t.cfg --pinmux_config tegra186-mb1-bct-pinmux-quill-p3310-1000-c03.cfg --pmic_config tegra186-mb1-bct-pmic-quill-p3310-1000-c04.cfg --pmc_config tegra186-mb1-bct-pad-quill-p3310-1000-c03.cfg --prod_config tegra186-mb1-bct-prod-quill-p3310-1000-c03.cfg --scr_config minimal_scr.cfg --scr_cold_boot_config mobile_scr.cfg --br_cmd_config tegra186-mb1-bct-bootrom-quill-p3310-1000-c03.cfg --dev_params emmc.cfg  --bins "mb2_bootloader nvtboot_recovery.bin; mts_preboot preboot_d15_prod_cr.bin; mts_bootpack mce_mts_d15_prod_cr.bin; bpmp_fw bpmp.bin; bpmp_fw_dtb tegra186-a02-bpmp-quill-p3310-1000-c04-00-te770d-ucm2.dtb; tlk tos.img; eks eks.img; bootloader_dtb tegra186-quill-p3310-1000-c03-00-base.dtb"  
saving flash command in /home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/flashcmd.txt
./tegraflash.py --bl nvtboot_recovery_cpu.bin --sdram_config P3310_A00_8GB_Samsung_8GB_lpddr4_204Mhz_A02_l4t.cfg --odmdata 0x1090000 --applet mb1_recovery_prod.bin --cmd "flash; reboot"  --cfg flash.xml --chip 0x18 --misc_config tegra186-mb1-bct-misc-si-l4t.cfg --pinmux_config tegra186-mb1-bct-pinmux-quill-p3310-1000-c03.cfg --pmic_config tegra186-mb1-bct-pmic-quill-p3310-1000-c04.cfg --pmc_config tegra186-mb1-bct-pad-quill-p3310-1000-c03.cfg --prod_config tegra186-mb1-bct-prod-quill-p3310-1000-c03.cfg --scr_config minimal_scr.cfg --scr_cold_boot_config mobile_scr.cfg --br_cmd_config tegra186-mb1-bct-bootrom-quill-p3310-1000-c03.cfg --dev_params emmc.cfg  --bins "mb2_bootloader nvtboot_recovery.bin; mts_preboot preboot_d15_prod_cr.bin; mts_bootpack mce_mts_d15_prod_cr.bin; bpmp_fw bpmp.bin; bpmp_fw_dtb tegra186-a02-bpmp-quill-p3310-1000-c04-00-te770d-ucm2.dtb; tlk tos.img; eks eks.img; bootloader_dtb tegra186-quill-p3310-1000-c03-00-base.dtb"  
copying boot image (boot.img.sb)... done.
copying partition layout (flash.xml.sb)... done.
copying kernel dtb (tegra186-quill-p3310-1000-c03-00-base.dtb.sb)... done.
saving flash parameters in /home/cortexia/Downloads/Jetpack_sysimg_copy/64_TX2/Linux_for_Tegra/bootloader/flash_parameters.txt
--bl nvtboot_recovery_cpu.bin --sdram_config P3310_A00_8GB_Samsung_8GB_lpddr4_204Mhz_A02_l4t.cfg --odmdata 0x1090000 --applet mb1_recovery_prod.bin --cmd "flash; reboot"  --cfg flash.xml --chip 0x18 --misc_config tegra186-mb1-bct-misc-si-l4t.cfg --pinmux_config tegra186-mb1-bct-pinmux-quill-p3310-1000-c03.cfg --pmic_config tegra186-mb1-bct-pmic-quill-p3310-1000-c04.cfg --pmc_config tegra186-mb1-bct-pad-quill-p3310-1000-c03.cfg --prod_config tegra186-mb1-bct-prod-quill-p3310-1000-c03.cfg --scr_config minimal_scr.cfg --scr_cold_boot_config mobile_scr.cfg --br_cmd_config tegra186-mb1-bct-bootrom-quill-p3310-1000-c03.cfg --dev_params emmc.cfg  --bins "mb2_bootloader nvtboot_recovery.bin; mts_preboot preboot_d15_prod_cr.bin; mts_bootpack mce_mts_d15_prod_cr.bin; bpmp_fw bpmp.bin; bpmp_fw_dtb tegra186-a02-bpmp-quill-p3310-1000-c04-00-te770d-ucm2.dtb; tlk tos.img; eks eks.img; bootloader_dtb tegra186-quill-p3310-1000-c03-00-base.dtb"  
* no-flash flag enabled. Exiting now... *

Can it work if only flash two devices?

I made my tests with only two devices ans it doesn’t work.

How about one device by multi_tegra_flash.sh?

the multi_tegra_flash.sh works fine with one two or three TX2 as long as I use the default system.img. When I try to use my custom system.img then it fail with one or two TX2.

Still blocked. Any ideas ?

hello marc.schnaebele,

please check Flashing and Booting the Target Device chapter, had you overwrite the default system.img with the backup image.
thanks

Yes of course I had like mentioned in the first post I know how to flash a single TX2 and I know how to flash multiple TX2 at the same time with the default system.img the only problem here is when I try to flash multiple TX2 at the same time with my backup image.

hello marc.schnaebele,

FYI,
we had verified we’re able to flash backup image on two Jetson-TX2 devices successfully.
suggest you share your detail steps for reference.
thanks