I ran the full process again, and indeed was able to flash the board. Looking further into it however I also found why, and it unfortunately is not a way to fix the issue.
Short version
MB2 applet does not start, but reading out device data fails silently and simply reuses previously read data. Trying to flash with a completely fresh extracted jetpack fails, and when removing bootloader/cvm.bin (the cvm eeprom data from the previous run) before flashing it also fails.
Booting initrd and everything else that does not rely on the MB2 applet works, which is why flashing can be successful.
Long version
What happens on the flashing side is the following:
l4t_initrd_flash.sh calls a few layers deep until it reaches the following tegraflash.py call (see log):
./tegraflash.py \
--chip "0x23" \
--applet "/path/to/Linux_for_Tegra/bootloader/mb1_t234_prod.bin" \
--skipuid \
# [...]
--bins "mb2_applet applet_t234.bin" \
--cmd "readfuses fuse_t234.bin fuse_t234.xml; dump eeprom cvm cvm.bin; dump try_custinfo custinfo_out.bin; reboot recovery"
tegraflash.py then calls the handlers for all commands one after the other:
do_readfusescalls into the t234 impl which first sets up BR communication and so on, but fails to boot into MB2 applet, then raises an error (“timeout in USB write”). This gets caught and printed, reading fuses is cancelled, but execution continues with the next command. This behavior is exclusive toreadfuses- all others fail early and cancel further command execution if an unexpected error is encountered.
All other commands think that an MB2 applet is running becausetegrarcm_v2 --ismb2appletsucceeds.do_dumpcalls into the t234 impl which does ategrarcm_v2call to read the eeprom. While this call returns success, no eeprom contents are read from the device or written to disk.do_dumpcalls into the t234 impl and tries to read the custinfo, which fails, is logged, and ignored.do_rebootalso seems to succeed, but I can not validate if rebooting actually happens.
dump cvm cvm.bin is suppsed to dump the CVM data to bootloader/cvm.bin, but when the device is in the state as described earlier, MB2 applet load fails and no CVM data is read, leaving data from a previous run (if available) intact. This is then used to flash the device.
For obvious reasons, this is not viable to use in production, especially when flashing multiple devices with potentially different SoM variants.
Could you try to reproduce the issue with either removing bootloader/cvm.bin before flashing or when using a fully fresh jetpack release which was not previously used for flashing?
Logs with bootloader/cvm.bin removed before flashing:
Flash Host Log
Please install the Secureboot package to use initrd flash for fused board
# Entry added by NVIDIA initrd flash tool
/home/pkauls/jetpack_r36.4_2/Linux_for_Tegra/tools/kernel_flash/tmp 127.0.0.1(rw,nohide,insecure,no_subtree_check,async,no_root_squash)
rpcbind: another rpcbind is already running. Aborting
Export list for localhost:
/home/pkauls/jetpack_r36.4_2/Linux_for_Tegra/tools/kernel_flash/tmp 127.0.0.1
/home/pkauls/jetpack_r36.4_2/Linux_for_Tegra/tools/kernel_flash/l4t_initrd_flash_internal.sh --no-flash bootloader/generic/cfg/flash_t234_qspi.xml jetson-orin-nano-devkit internal
************************************
* *
* Step 1: Generate flash packages *
* *
************************************
Create folder to store images to flash
Generate image for internal storage devices
Generate images to be flashed
ADDITIONAL_DTB_OVERLAY="" /home/pkauls/jetpack_r36.4_2/Linux_for_Tegra/flash.sh --no-flash --sign jetson-orin-nano-devkit internal
###############################################################################
# L4T BSP Information:
# R36 , REVISION: 4.0
# User release: 0.0
###############################################################################
ECID is 0x80012344705DD3570800000001028140
copying device_config(/home/pkauls/jetpack_r36.4_2/Linux_for_Tegra/bootloader/generic/BCT/tegra234-mb1-bct-device-p3767-0000.dts)... done.
copying misc_config(/home/pkauls/jetpack_r36.4_2/Linux_for_Tegra/bootloader/generic/BCT/tegra234-mb1-bct-misc-p3767-0000.dts)... done.
copying emc_fuse_dev_params(/home/pkauls/jetpack_r36.4_2/Linux_for_Tegra/bootloader/generic/BCT/tegra234-br-bct-diag-boot.dts)... done.
Existing emcfuse(/home/pkauls/jetpack_r36.4_2/Linux_for_Tegra/bootloader/fuse_t234.xml) reused.
./tegraflash.py --chip "0x23" --applet "/home/pkauls/jetpack_r36.4_2/Linux_for_Tegra/bootloader/mb1_t234_prod.bin" --skipuid --cfg readinfo_t234_min_prod.xml --dev_params tegra234-br-bct-diag-boot.dts --device_config tegra234-mb1-bct-device-p3767-0000.dts --misc_config tegra234-mb1-bct-misc-p3767-0000.dts --bins "mb2_applet applet_t234.bin" --cmd "readfuses fuse_t234.bin fuse_t234.xml; dump eeprom cvm cvm.bin; dump try_custinfo custinfo_out.bin; reboot recovery"
Welcome to Tegra Flash
version 1.0.0
Type ? or help for help and q or quit to exit
Use ! to execute system commands
[ 0.0195 ] Reading fuses
[ 0.0199 ] tegrarcm_v2 --chip 0x23 0 --ismb2applet
[ 0.0201 ] File rcm_state open failed
[ 0.0203 ] ERROR: failed to read rcm_state
[ 0.0203 ]
[ 0.0210 ] tegrasign_v3.py --key None --getmode mode.txt
[ 0.0211 ] Assuming zero filled SBK key
[ 0.0206 ] Pre-processing config: tegra234-mb1-bct-device-p3767-0000.dts
[ 0.0264 ] Pre-processing config: tegra234-mb1-bct-misc-p3767-0000.dts
[ 0.0346 ] Parsing partition layout
[ 0.0349 ] tegraparser_v2 --pt readinfo_t234_min_prod.xml.tmp
[ 0.0357 ] Kernel DTB used: None
[ 0.0358 ] WARNING: dce base dtb is not provided
[ 0.0358 ] Parsing partition layout
[ 0.0361 ] tegraparser_v2 --pt readinfo_t234_min_prod.xml.tmp
[ 0.0366 ] Creating list of images to be signed
[ 0.0369 ] tegrahost_v2 --chip 0x23 0 --partitionlayout readinfo_t234_min_prod.xml.bin --list images_list.xml zerosbk
[ 0.0372 ] MB1: Nvheader already present is mb1_t234_prod_aligned.bin
[ 0.0382 ] Header already present for mb1_t234_prod_aligned_sigheader.bin
[ 0.0385 ] MB1: Nvheader already present is mb1_t234_prod_aligned.bin
[ 0.0430 ] Header already present for mb1_t234_prod_aligned_sigheader.bin
[ 0.0436 ] MB1: Nvheader already present is psc_bl1_t234_prod_aligned.bin
[ 0.0481 ] Header already present for psc_bl1_t234_prod_aligned_sigheader.bin
[ 0.0482 ] adding BCH for mb2_t234_aligned.bin
[ 0.0508 ] MB1: Nvheader already present is psc_bl1_t234_prod_aligned.bin
[ 0.0624 ] Header already present for psc_bl1_t234_prod_aligned_sigheader.bin
[ 0.0625 ] adding BCH for mb2_t234_aligned.bin
[ 0.0766 ] Filling MB1 storage info
[ 0.0766 ] Parsing dev params for multi chains
[ 0.0826 ] Generating br-bct
[ 0.0829 ] Updating dev and MSS params in BR BCT
[ 0.0829 ] tegrabct_v2 --dev_param tegra234-br-bct-diag-boot_cpp.dtb --brbct br_bct.cfg --chip 0x23 0
[ 0.0834 ] Updating bl info
[ 0.0838 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --updateblinfo readinfo_t234_min_prod.xml.bin
[ 0.0840 ] WARNING: boot chain is not completed. set to 0
[ 0.0847 ] Generating signatures
[ 0.0853 ] tegrasign_v3.py --key None --list images_list.xml --pubkeyhash pub_key.key --sha sha512
[ 0.0854 ] Assuming zero filled SBK key
[ 0.0940 ] Warning: pub_key.key is not found
[ 0.0935 ] Parsing dev params for multi chains
[ 0.0935 ] Generating br-bct
[ 0.0938 ] Updating dev and MSS params in BR BCT
[ 0.0939 ] tegrabct_v2 --dev_param tegra234-br-bct-diag-boot_cpp.dtb --brbct br_bct.cfg --chip 0x23 0
[ 0.0944 ] Updating bl info
[ 0.0947 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --updateblinfo readinfo_t234_min_prod.xml.bin --updatesig images_list_signed.xml
[ 0.0950 ] WARNING: boot chain is not completed. set to 0
[ 0.0960 ] Generating SHA2 Hash
[ 0.0973 ] Sha saved in br_bct_BR.sha
[ 0.0968 ] Get Signed section of bct
[ 0.0970 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --listbct bct_list.xml
[ 0.0975 ] Signing BCT
[ 0.0982 ] tegrasign_v3.py --key None --list bct_list.xml --pubkeyhash pub_key.key --sha sha512
[ 0.0983 ] Assuming zero filled SBK key
[ 0.1000 ] Sha saved in br_bct_BR.sha
[ 0.1002 ] Warning: pub_key.key is not found
[ 0.0996 ] Updating BCT with signature
[ 0.0999 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --updatesig bct_list_signed.xml
[ 0.1001 ] Offset :4608 Len :3584
[ 0.1005 ] Generating SHA2 Hash
[ 0.1011 ] tegrasign_v3.py --key None --list bct_list.xml --sha sha512
[ 0.1012 ] Assuming zero filled SBK key
[ 0.1012 ] Assuming zero filled SBK key
[ 0.1030 ] Sha saved in br_bct_BR.sha
[ 0.1025 ] Updating BCT with SHA2 Hash
[ 0.1028 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --updatesha bct_list_signed.xml
[ 0.1031 ] Offset :4608 Len :3584
[ 0.1033 ] Offset :68 Len :8124
[ 0.1035 ] Generating coldboot mb1-bct
[ 0.1038 ] tegrabct_v2 --chip 0x23 0 --mb1bct mb1_cold_boot_bct.cfg --misc tegra234-mb1-bct-misc-p3767-0000_cpp.dtb --device tegra234-mb1-bct-device-p3767-0000_cpp.dtb
[ 0.1041 ] MB1-BCT version: 0.13
[ 0.1057 ] Parsing config file :tegra234-mb1-bct-device-p3767-0000_cpp.dtb
[ 0.1059 ] Added Platform Config 9 data with size :- 100
[ 0.1059 ]
[ 0.1059 ] Updating mb1-bct with firmware information
[ 0.1062 ] tegrabct_v2 --chip 0x23 0 --mb1bct mb1_cold_boot_bct_MB1.bct --updatefwinfo readinfo_t234_min_prod.xml.bin
[ 0.1073 ] tegrahost_v2 --chip 0x23 0 --align mb1_cold_boot_bct_MB1_aligned.bct
[ 0.1076 ] Generating SHA2 Hash for mb1bct
[ 0.1091 ] Sha saved in mb1_cold_boot_bct_MB1_aligned.sha
[ 0.1099 ] Sha saved in mb1_cold_boot_bct_MB1.sha
[ 0.1096 ] tegrahost_v2 --chip 0x23 0 --magicid MBCT --appendsigheader mb1_cold_boot_bct_MB1_aligned.bct zerosbk
[ 0.1099 ] adding BCH for mb1_cold_boot_bct_MB1_aligned.bct
[ 0.1112 ] tegrasign_v3.py --key None --list mb1_cold_boot_bct_MB1_aligned_sigheader.bct_list.xml --pubkeyhash pub_key.key --sha sha512
[ 0.1113 ] Assuming zero filled SBK key
[ 0.1124 ] Warning: pub_key.key is not found
[ 0.1121 ] tegrahost_v2 --chip 0x23 0 --updatesigheader mb1_cold_boot_bct_MB1_aligned_sigheader.bct.encrypt mb1_cold_boot_bct_MB1_aligned_sigheader.bct.hash zerosbk
[ 0.1129 ] Generating recovery mb1-bct
[ 0.1132 ] tegrabct_v2 --chip 0x23 0 --mb1bct mb1_bct.cfg --misc tegra234-mb1-bct-misc-p3767-0000_cpp.dtb --device tegra234-mb1-bct-device-p3767-0000_cpp.dtb
[ 0.1134 ] MB1-BCT version: 0.13
[ 0.1150 ] Parsing config file :tegra234-mb1-bct-device-p3767-0000_cpp.dtb
[ 0.1152 ] Added Platform Config 9 data with size :- 100
[ 0.1152 ]
[ 0.1152 ] Updating mb1-bct with firmware information
[ 0.1155 ] tegrabct_v2 --chip 0x23 0 --mb1bct mb1_bct_MB1.bct --recov --updatefwinfo readinfo_t234_min_prod.xml.bin
[ 0.1165 ] tegrahost_v2 --chip 0x23 0 --align mb1_bct_MB1_aligned.bct
[ 0.1169 ] Generating SHA2 Hash for mb1bct
[ 0.1183 ] Sha saved in mb1_bct_MB1_aligned.sha
[ 0.1191 ] Sha saved in mb1_bct_MB1.sha
[ 0.1189 ] tegrahost_v2 --chip 0x23 0 --magicid MBCT --appendsigheader mb1_bct_MB1_aligned.bct zerosbk
[ 0.1191 ] adding BCH for mb1_bct_MB1_aligned.bct
[ 0.1204 ] tegrasign_v3.py --key None --list mb1_bct_MB1_aligned_sigheader.bct_list.xml --pubkeyhash pub_key.key --sha sha512
[ 0.1204 ] Assuming zero filled SBK key
[ 0.1215 ] Warning: pub_key.key is not found
[ 0.1212 ] tegrahost_v2 --chip 0x23 0 --updatesigheader mb1_bct_MB1_aligned_sigheader.bct.encrypt mb1_bct_MB1_aligned_sigheader.bct.hash zerosbk
[ 0.1220 ] Info: Skip generating mem_bct because sdram_config is not defined
[ 0.1220 ] Info: Skip generating mem_bct because sdram_config is not defined
[ 0.1220 ] Copying signatures
[ 0.1224 ] tegrahost_v2 --chip 0x23 0 --partitionlayout readinfo_t234_min_prod.xml.bin --updatesig images_list_signed.xml
[ 0.1268 ] mb1_t234_prod_aligned_sigheader.bin.encrypt filename is from images_list
[ 0.1269 ] psc_bl1_t234_prod_aligned_sigheader.bin.encrypt filename is from images_list
[ 0.1269 ] Boot Rom communication
[ 0.1272 ] tegrarcm_v2 --new_session --chip 0x23 0 --uid --download bct_br br_bct_BR.bct --download mb1 mb1_t234_prod_aligned_sigheader.bin.encrypt --download psc_bl1 psc_bl1_t234_prod_aligned_sigheader.bin.encrypt --download bct_mb1 mb1_bct_MB1_sigheader.bct.encrypt
[ 0.1275 ] BR_CID: 0x80012344705DD3570800000001028140
[ 0.1283 ] Sending bct_br
[ 0.1284 ] Sending mb1
[ 0.1288 ] Sending psc_bl1
[ 0.1392 ] Sending bct_mb1
[ 0.1449 ] Boot Rom communication completed
[ 0.1456 ] tegrahost_v2 --chip 0x23 0 --align applet_t234_aligned.bin
[ 0.1464 ] tegrahost_v2 --chip 0x23 0 --magicid MB2A --appendsigheader applet_t234_aligned.bin zerosbk
[ 0.1467 ] adding BCH for applet_t234_aligned.bin
[ 0.1558 ] tegrasign_v3.py --key None --list applet_t234_aligned_sigheader.bin_list.xml --pubkeyhash pub_key.key --sha sha512
[ 0.1559 ] Assuming zero filled SBK key
[ 0.1575 ] Warning: pub_key.key is not found
[ 0.1573 ] tegrahost_v2 --chip 0x23 0 --updatesigheader applet_t234_aligned_sigheader.bin.encrypt applet_t234_aligned_sigheader.bin.hash zerosbk
[ 0.1586 ] Sending mb2_applet...
[ 0.1589 ] tegrarcm_v2 --chip 0x23 0 --pollbl --download applet applet_t234_sigheader.bin.encrypt
[ 0.1592 ] BL: version 1.4.0.4-t234-54845784-e89ea9bc last_boot_error: 656573201
[ 0.3689 ] Sending applet
[ 0.3693 ] ERROR: might be timeout in USB write.
Error: Return value 3
Command tegrarcm_v2 --chip 0x23 0 --pollbl --download applet applet_t234_sigheader.bin.encrypt
[ 0.4651 ] tegrarcm_v2 --chip 0x23 0 --ismb2applet
[ 0.4665 ] tegrarcm_v2 --chip 0x23 0 --ismb2applet
[ 0.4674 ] Retrieving board information
[ 0.4679 ] tegrarcm_v2 --chip 0x23 0 --oem platformdetails chip chip_info.bin
[ 0.4685 ] Retrieving EEPROM data
[ 0.4685 ] tegrarcm_v2 --oem platformdetails eeprom cvm /home/pkauls/jetpack_r36.4_2/Linux_for_Tegra/bootloader/cvm.bin --chip 0x23 0
[ 0.4694 ] tegrarcm_v2 --chip 0x23 0 --ismb2applet
[ 0.4703 ] tegrarcm_v2 --chip 0x23 0 --ismb2applet
[ 0.4707 ] Dumping customer Info
[ 0.4711 ] tegrarcm_v2 --chip 0x23 0 --oem dump bct tmp.bct
[ 0.4719 ] tegrabct_v2 --brbct tmp.bct --chip 0x23 0 --custinfo /home/pkauls/jetpack_r36.4_2/Linux_for_Tegra/bootloader/custinfo_out.bin
[ 0.4722 ] File tmp.bct open failed
[ 0.4724 ] Error: try getting custinfo fail. Moving on
[ 0.4724 ] Rebooting to recovery mode
[ 0.4728 ] tegrarcm_v2 --chip 0x23 0 --ismb2
[ 0.4732 ] Rebooting to recovery mode
[ 0.4735 ] tegrarcm_v2 --chip 0x23 0 --reboot recovery
Parsing board information failed.
Error: failed to generate images
Cleaning up...
Flash Device UART Log
jetson@jetson-dk-7:~$ sudo reboot forced-recovery
[sudo] password for jetson:
[ 152.687146] tegra-ivc-bus bc00000.rtcpu:ivc-bus:echo@0: ivc channel driver missing
[ 152.687158] tegra-ivc-bus bc00000.rtcpu:ivc-bus:dbg@1: ivc channel driver missing
[ 152.687164] tegra-ivc-bus bc00000.rtcpu:ivc-bus:dbg@2: ivc channel driver missing
[ 152.687171] tegra-ivc-bus bc00000.rtcpu:ivc-bus:diag@5: ivc channel driver missing
[ 152.895796] nvgpu: 17000000.gpu pmu_wait_message_cond:664 [ERR] PMU wait timeout expired.
[ 152.895803] nvgpu: 17000000.gpu pmu_wait_message_cond:664 [ERR] PMU wait timeout expired.
[ 152.896467] nvgpu: 17000000.gpu ga10b_intr_log_pending_intrs:306 [ERR] Pending TOP[0]: 0x00000004, LEAF[4]: 0x11000000
[ 152.897897] arm-smmu 12000000.iommu: disabling translation
[ 152.898033] arm-smmu 10000000.iommu: disabling translation
[ 152.898091] arm-smmu 8000000.iommu: disabling translation
[ 152.898315] reboot: Restarting system with command 'forced-recovery'
▒▒▒▒Shutdown state requested 1
Rebooting system ...
▒▒
[0014.940] I> MB1 (version: 1.4.0.4-t234-54845784-e89ea9bc)
[0014.945] I> t234-A01-0-Silicon (0x12347) Prod
[0014.949] I> Boot-mode : BPMP Diagnostic
[0014.953] I> Entry timestamp: 0x00000000
[0014.957] I> last_boot_error: 0x0
[0014.960] I> BR-BCT: preprod_dev_sign: 0
[0014.964] I> rst_source: 0xb, rst_level: 0x1
[0014.968] I> Task: Initialize MB2 params
[0014.972] I> MB2-params @ 0x40060000
[0014.976] I> Task: Crypto init
[0014.979] I> Task: Perform MB1 KAT tests
[0014.983] I> Task: NVRNG health check
[0014.986] I> NVRNG: Health check success
[0014.990] I> Task: Secure debug controls
[0014.994] I> Task: strap war set
[0014.997] I> Task: Program NV master stream id
[0015.001] I> Task: Verify boot mode
[0015.007] I> Task: Alias fuses
[0015.010] W> FUSE_ALIAS: Fuse alias on production fused part is not supported.
[0015.017] I> Task: Print SKU type
[0015.020] I> FUSE_OPT_CCPLEX_CLUSTER_DISABLE = 0x000001c8
[0015.025] I> FUSE_OPT_GPC_DISABLE = 0x00000002
[0015.030] I> FUSE_OPT_TPC_DISABLE = 0x000000f0
[0015.034] I> FUSE_OPT_DLA_DISABLE = 0x00000003
[0015.038] I> FUSE_OPT_PVA_DISABLE = 0x00000001
[0015.042] I> FUSE_OPT_NVENC_DISABLE = 0x00000001
[0015.047] I> FUSE_OPT_NVDEC_DISABLE = 0x00000000
[0015.051] I> FUSE_OPT_FSI_DISABLE = 0x00000001
[0015.056] I> FUSE_OPT_EMC_DISABLE = 0x0000000c
[0015.060] I> FUSE_BOOTROM_PATCH_VERSION = 0x7
[0015.064] I> FUSE_PSCROM_PATCH_VERSION = 0x7
[0015.068] I> FUSE_OPT_ADC_CAL_FUSE_REV = 0x2
[0015.072] I> FUSE_SKU_INFO_0 = 0xd5
[0015.076] I> FUSE_OPT_SAMPLE_TYPE_0 = 0x3 PS
[0015.080] I> FUSE_PACKAGE_INFO_0 = 0x2
[0015.083] I> SKU: Prod
[0015.086] I> Task: Boost clocks
[0015.089] I> Initializing PLLC2 for AXI_CBB.
[0015.093] I> AXI_CBB : src = 35, divisor = 0
[0015.097] I> Task: Voltage monitor
[0015.100] I> VMON: Vmon re-calibration and fine tuning done
[0015.106] I> Task: UPHY init
[0015.109] W> UPHY: UPHY lane info table is empty in MB1 BCT.
[0015.115] I> Task: Boot device init
[0015.118] I> Boot_device: RCM
[0015.121] I> USB configuration success
[0015.125] I> Task: TSC init
[0015.128] I> Task: Enable WDT 5th expiry
[0015.131] I> Task: I2C register
[0015.134] I> Task: Set I2C bus freq
[0015.138] I> Task: Reset FSI
[0015.140] I> Task: Enable clock-mon
[0015.144] I> FMON: Fmon re-programming done
[0015.149] I> Task: Mapper init
[0015.151] I> Task: MB1 fixed firewalls
[0015.158] W> Firewall readback mismatch
[0015.163] I> Task: Load MB2/Applet/FSKP
[0015.167] I> Loading MB2 Applet
[0015.170] I> Slot: 1
[0015.172] E> BLOCK_DEV: Failed to open blockdev.
[0015.177] E> LOADER: Failed to open blockdev 0(0).
[0015.182] E> LOADER: Failed to get storage info for binary 21 from loader.
[0015.189] C> LOADER: Could not read binary 21.
[0015.194] E> Failed to load MB2
[0015.197] C> Task 0x46 failed (err: 0x27228311)
[0015.201] E> Top caller module: MB2_PARAMS, error module: LOADER, reason: 0x11, aux_info: 0x83
[0015.210] C> Boot Info Table status dump :
0111100000111000110111111111000000011110000000000000011000001
▒▒