Orin Flash Failure

Autonomous Machines Jetson & Embedded Systems Jetson AGX Orin
1

My Orin died when trying to update to the latest jetpack from the command line via ‘apt dist-upgrade’. After reboot I can get press escape to enter a setup screen but I cannot boot. I have been attempting to flash the system but numerous failures have halted progress and I’m looking for ideas.
Things to note:

  1. Orin is 64GB dev kit.
  2. Yes, I can see the device on the USB as Bus 001 Device 014: ID 0955:7023 NVidia Corp.
  3. First failure is:
ERROR: Flash Jetson Linux - flash: [ 0.0986 ] ERROR: /misc/tsc_controls/tsc_locking_config is not supported
  1. I have tried numerous host systems, Ubuntu 22.04, Ubuntu 18.04 VM on 22.04, and just Ubuntu 18.04. In all cases the failure is the same.
  2. I am connected to the USB port and SDK manager definitely “see’s” my Orin is in recover mode.
  3. I am connected to the USB/TTY port for debug and during normal boot I can see serial scrolling text. The last lines during normal boot-up attempts are:
**  WARNING: Test Key is used.  **

L4TLauncher: Attempting Direct Boot
EFI stub: Booting Linux Kernel...
EFI stub: Using DTB from configuration table
EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path
EFI stub: Exiting boot services and installing virtual address map...

Also, during recover-mode and flashing this ttyACM0 appears to be silent.

As I can get into the setup and choose the boot manager is there a way to flash onto an SD-CARD or NVME via the host and boot from that device? Using Etcher or similar tool?

Dump from SDK Manager here:

08:12:34 DEBUG: running command < /tmp/device_mode_host_setup.sh >
08:12:34 INFO: exec_command: /tmp/device_mode_host_setup.sh
08:12:34 INFO: Found route default wlp111s0, setting up device mode
08:12:34 INFO: [host] [sudo] password for apt:
08:12:34 INFO: net.ipv4.ip_forward = 1
08:12:34 INFO: Device mode on host successfully set with DNS 8.8.8.8!
08:12:34 INFO: command finished successfully
08:12:34 DEBUG: running command < true >
08:12:34 INFO: command finished successfully
08:12:34 DEBUG: Flash Jetson Linux - flash: using adapter to install NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_AGX_ORIN_TARGETS to /home/apt/nvidia/nvidia_sdk/JetPack_5.1.1_Linux_JETSON_AGX_ORIN_TARGETS
08:12:34 INFO: Flash Jetson Linux - flash: current working directory is /home/apt/nvidia/nvidia_sdk/JetPack_5.1.1_Linux_JETSON_AGX_ORIN_TARGETS/Linux_for_Tegra
08:12:34 INFO: Flash Jetson Linux - flash: sudo ./nvsdkmanager_flash.sh
08:12:34 INFO: Flash Jetson Linux - flash: Defaulting to autoflash
08:12:34 INFO: Flash Jetson Linux - flash: *** Checking ONLINE mode ... OK.
08:12:35 INFO: Flash Jetson Linux - flash: *** Checking target board connection ... 1 connections found.
08:12:38 INFO: Flash Jetson Linux - flash: *** Reading ECID ... FUSELEVEL=fuselevel_production hwchipid=0x23 bootauth=NS
08:12:38 INFO: Flash Jetson Linux - flash: *** Reading EEPROM ... "/home/apt/nvidia/nvidia_sdk/JetPack_5.1.1_Linux_JETSON_AGX_ORIN_TARGETS/Linux_for_Tegra/bootloader/tegraflash.py" --chip 0x23 --applet "/home/apt/nvidia/nvidia_sdk/JetPack_5.1.1_Linux_JETSON_AGX_ORIN_TARGETS/Linux_for_Tegra/bootloader/mb1_t234_prod.bin" --cfg readinfo_t234_min_prod.xml --dev_params tegra234-br-bct-diag-boot.dts --device_config tegra234-mb1-bct-device-p3701-0000.dts --misc_config tegra234-mb1-bct-misc-p3701-0000.dts --bins "mb2_applet applet_t234.bin" --skipuid --cmd "dump eeprom cvm cvm.bin; reboot recovery"
08:12:38 INFO: Flash Jetson Linux - flash: [ 0.0114 ] tegrarcm_v2 --chip 0x23 0 --ismb2applet
08:12:38 INFO: Flash Jetson Linux - flash: [ 0.0120 ] File rcm_state open failed
08:12:38 INFO: Flash Jetson Linux - flash: [ 0.0257 ] MB1: Nvheader already present is mb1_t234_prod_aligned.bin
08:12:38 INFO: Flash Jetson Linux - flash: [ 0.0264 ] MB1: Nvheader already present is mb1_t234_prod_aligned.bin
08:12:39 INFO: Flash Jetson Linux - flash: [ 0.0299 ] Header already present for mb1_t234_prod_aligned_sigheader.bin
08:12:39 INFO: Flash Jetson Linux - flash: [ 0.0413 ] MB1: Nvheader already present is psc_bl1_t234_prod_aligned.bin
08:12:39 INFO: Flash Jetson Linux - flash: [ 0.0481 ] Header already present for psc_bl1_t234_prod_aligned_sigheader.bin
08:12:39 INFO: Flash Jetson Linux - flash: [ 0.0483 ] adding BCH for mb2_t234_aligned.bin
08:12:39 INFO: Flash Jetson Linux - flash: [ 0.0597 ] Header already present for psc_bl1_t234_prod_aligned_sigheader.bin
08:12:39 INFO: Flash Jetson Linux - flash: [ 0.0599 ] adding BCH for mb2_t234_aligned.bin
08:12:39 INFO: Flash Jetson Linux - flash: [ 0.0769 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --updateblinfo readinfo_t234_min_prod.xml.bin
08:12:39 INFO: Flash Jetson Linux - flash: [ 0.0791 ] Assuming zero filled SBK key
08:12:39 INFO: Flash Jetson Linux - flash: [ 0.0880 ] WARNING: boot chain is not completed. set to 0
08:12:39 INFO: Flash Jetson Linux - flash: [ 0.0909 ] Assuming zero filled SBK key
08:12:39 INFO: Flash Jetson Linux - flash: [ 0.0928 ] Offset :4608 Len :3584
08:12:39 INFO: Flash Jetson Linux - flash: [ 0.0942 ] Assuming zero filled SBK key
08:12:39 INFO: Flash Jetson Linux - flash: [ 0.0963 ] Offset :68 Len :8124
08:12:39 ERROR: Flash Jetson Linux - flash: [ 0.0986 ] ERROR: /misc/tsc_controls/tsc_locking_config is not supported
08:12:39 INFO: Flash Jetson Linux - flash: [ 0.1022 ] adding BCH for mb1_cold_boot_bct_MB1_aligned.bct
08:12:39 INFO: Flash Jetson Linux - flash: [ 0.1040 ] Assuming zero filled SBK key
08:12:39 INFO: Flash Jetson Linux - flash: [ 0.1065 ] MB1-BCT version: 0.10
08:12:39 ERROR: Flash Jetson Linux - flash: [ 0.1075 ] ERROR: /misc/tsc_controls/tsc_locking_config is not supported
08:12:39 INFO: Flash Jetson Linux - flash: [ 0.1111 ] adding BCH for mb1_bct_MB1_aligned.bct
08:12:39 INFO: Flash Jetson Linux - flash: [ 0.1128 ] Assuming zero filled SBK key
08:12:39 INFO: Flash Jetson Linux - flash: [ 0.1190 ] tegrarcm_v2 --new_session --chip 0x23 0 --uid --download bct_br br_bct_BR.bct --download mb1 mb1_t234_prod_aligned_sigheader.bin.encrypt --download psc_bl1 psc_bl1_t234_prod_aligned_sigheader.bin.encrypt --download bct_mb1 mb1_bct_MB1_sigheader.bct.encrypt
08:12:39 INFO: Flash Jetson Linux - flash: [ 0.1195 ] BR_CID: 0x80012344705DD5DD2C00000011FE8100
08:12:40 INFO: Flash Jetson Linux - flash: [ 0.7263 ] Sending bct_br
08:12:45 ERROR: Flash Jetson Linux - flash: [ 1.5416 ] ERROR: might be timeout in USB write.
08:12:45 INFO: Flash Jetson Linux - flash: Command tegrarcm_v2 --new_session --chip 0x23 0 --uid --download bct_br br_bct_BR.bct --download mb1 mb1_t234_prod_aligned_sigheader.bin.encrypt --download psc_bl1 psc_bl1_t234_prod_aligned_sigheader.bin.encrypt --download bct_mb1 mb1_bct_MB1_sigheader.bct.encrypt
08:12:45 ERROR: Flash Jetson Linux - flash: --- Error: Reading board information failed.
08:12:45 ERROR: Flash Jetson Linux - flash: command terminated with error
08:12:45 SUMMARY: DateTime Target Setup - target: Depends on failed component

Dump from flash.sh here:

###############################################################################
# L4T BSP Information:
# R35 , REVISION: 3.1
# User release: 0.0
###############################################################################
# Target Board Information:
# Name: jetson-agx-orin-devkit, Board Family: t186ref, SoC: Tegra 234, 
# OpMode: production, Boot Authentication: NS, 
# Disk encryption: disabled ,
###############################################################################
copying emc_fuse_dev_params(/home/apt/nvidia/nvidia_sdk/JetPack_5.1.1_Linux_JETSON_AGX_ORIN_TARGETS/Linux_for_Tegra/bootloader/t186ref/BCT/tegra234-br-bct-diag-boot.dts)... done.
copying device_config(/home/apt/nvidia/nvidia_sdk/JetPack_5.1.1_Linux_JETSON_AGX_ORIN_TARGETS/Linux_for_Tegra/bootloader/t186ref/BCT/tegra234-mb1-bct-device-p3701-0000.dts)... done.
copying misc_config(/home/apt/nvidia/nvidia_sdk/JetPack_5.1.1_Linux_JETSON_AGX_ORIN_TARGETS/Linux_for_Tegra/bootloader/t186ref/BCT/tegra234-mb1-bct-misc-p3701-0000.dts)... done.
./tegraflash.py --chip 0x23 --applet "/home/apt/nvidia/nvidia_sdk/JetPack_5.1.1_Linux_JETSON_AGX_ORIN_TARGETS/Linux_for_Tegra/bootloader/mb1_t234_prod.bin" --skipuid --cfg readinfo_t234_min_prod.xml --dev_params tegra234-br-bct-diag-boot.dts --device_config tegra234-mb1-bct-device-p3701-0000.dts --misc_config tegra234-mb1-bct-misc-p3701-0000.dts --bins "mb2_applet applet_t234.bin" --cmd "dump eeprom cvm cvm.bin; dump custinfo custinfo_out.bin; reboot recovery" 
Welcome to Tegra Flash
version 1.0.0
Type ? or help for help and q or quit to exit
Use ! to execute system commands
 
[   0.0105 ] tegrarcm_v2 --chip 0x23 0 --ismb2applet
[   0.0110 ] File rcm_state open failed
[   0.0112 ] ERROR: failed to read rcm_state
[   0.0112 ] 
[   0.0120 ] tegrasign_v3.py --key None --getmode mode.txt
[   0.0122 ] Assuming zero filled SBK key
[   0.0116 ] Pre-processing config: tegra234-mb1-bct-device-p3701-0000.dts
[   0.0150 ] Pre-processing config: tegra234-mb1-bct-misc-p3701-0000.dts
[   0.0201 ] Parsing partition layout
[   0.0207 ] tegraparser_v2 --pt readinfo_t234_min_prod.xml.tmp
[   0.0216 ] Kernel DTB used: None
[   0.0217 ] WARNING: dce base dtb is not provided

[   0.0217 ] Parsing partition layout
[   0.0222 ] tegraparser_v2 --pt readinfo_t234_min_prod.xml.tmp
[   0.0230 ] Creating list of images to be signed
[   0.0236 ] tegrahost_v2 --chip 0x23 0 --partitionlayout readinfo_t234_min_prod.xml.bin --list images_list.xml zerosbk
[   0.0241 ] MB1: Nvheader already present is mb1_t234_prod_aligned.bin
[   0.0247 ] Header already present for mb1_t234_prod_aligned_sigheader.bin
[   0.0249 ] MB1: Nvheader already present is mb1_t234_prod_aligned.bin
[   0.0282 ] Header already present for mb1_t234_prod_aligned_sigheader.bin
[   0.0310 ] MB1: Nvheader already present is psc_bl1_t234_prod_aligned.bin
[   0.0344 ] Header already present for psc_bl1_t234_prod_aligned_sigheader.bin
[   0.0346 ] adding BCH for mb2_t234_aligned.bin
[   0.0364 ] MB1: Nvheader already present is psc_bl1_t234_prod_aligned.bin
[   0.0455 ] Header already present for psc_bl1_t234_prod_aligned_sigheader.bin
[   0.0457 ] adding BCH for mb2_t234_aligned.bin
[   0.0563 ] Filling MB1 storage info
[   0.0563 ] Parsing dev params for multi chains
[   0.0597 ] Generating br-bct
[   0.0602 ] Updating dev and MSS params in BR BCT
[   0.0603 ] tegrabct_v2 --dev_param tegra234-br-bct-diag-boot_cpp.dtb --brbct br_bct.cfg --chip 0x23 0
[   0.0610 ] Updating bl info
[   0.0616 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --updateblinfo readinfo_t234_min_prod.xml.bin
[   0.0621 ] WARNING: boot chain is not completed. set to 0
[   0.0626 ] Generating signatures
[   0.0634 ] tegrasign_v3.py --key None --list images_list.xml --pubkeyhash pub_key.key --sha sha512
[   0.0635 ] Assuming zero filled SBK key
[   0.0702 ] Warning: pub_key.key is not found
[   0.0696 ] Parsing dev params for multi chains
[   0.0696 ] Generating br-bct
[   0.0701 ] Updating dev and MSS params in BR BCT
[   0.0701 ] tegrabct_v2 --dev_param tegra234-br-bct-diag-boot_cpp.dtb --brbct br_bct.cfg --chip 0x23 0
[   0.0708 ] Updating bl info
[   0.0714 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --updateblinfo readinfo_t234_min_prod.xml.bin --updatesig images_list_signed.xml
[   0.0719 ] WARNING: boot chain is not completed. set to 0
[   0.0725 ] Get Signed section of bct
[   0.0730 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --listbct bct_list.xml
[   0.0737 ] Signing BCT
[   0.0745 ] tegrasign_v3.py --key None --list bct_list.xml --pubkeyhash pub_key.key --sha sha512
[   0.0747 ] Assuming zero filled SBK key
[   0.0763 ] Sha saved in br_bct_BR.sha
[   0.0766 ] Warning: pub_key.key is not found
[   0.0758 ] Updating BCT with signature
[   0.0763 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --updatesig bct_list_signed.xml
[   0.0768 ] Offset :4608 Len :3584
[   0.0771 ] Generating SHA2 Hash
[   0.0779 ] tegrasign_v3.py --key None --list bct_list.xml --sha sha512
[   0.0780 ] Assuming zero filled SBK key
[   0.0780 ] Assuming zero filled SBK key
[   0.0796 ] Sha saved in br_bct_BR.sha
[   0.0791 ] Updating BCT with SHA2 Hash
[   0.0795 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --updatesha bct_list_signed.xml
[   0.0800 ] Offset :4608 Len :3584
[   0.0802 ] Offset :68 Len :8124
[   0.0804 ] Generating coldboot mb1-bct
[   0.0809 ] tegrabct_v2 --chip 0x23 0 --mb1bct mb1_cold_boot_bct.cfg --misc tegra234-mb1-bct-misc-p3701-0000_cpp.dtb --device tegra234-mb1-bct-device-p3701-0000_cpp.dtb
[   0.0815 ] MB1-BCT version: 0.10
[   0.0816 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_UNUSED5/ is not supported
[   0.0820 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_UNUSED5/ is not supported
[   0.0823 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_UNUSED5/ is not supported
[   0.0827 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_MCE_COVERAGE/ is not supported
[   0.0830 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_MCE_COVERAGE/ is not supported
[   0.0830 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_MCE_COVERAGE/ is not supported
[   0.0830 ] ERROR: /misc/tsc_controls/tsc_locking_config is not supported
[   0.0830 ] ERROR: /misc/tsc_controls/tsc_locking_diff_configuration is not supported
[   0.0830 ] ERROR: /misc/tsc_controls/tsc_locking_ref_frequency_configuration is not supported
[   0.0830 ] ERROR: /misc/tsc_controls/tsc_locking_control is not supported
[   0.0830 ] ERROR: /misc/tsc_controls/tsc_locking_adjust_configuration is not supported
[   0.0830 ] ERROR: /misc/tsc_controls/tsc_locking_fast_adjust_configuration is not supported
[   0.0830 ] ERROR: /misc/tsc_controls/tsc_locking_adjust_delta_control is not supported
[   0.0830 ] ERROR: /misc/tsc_controls/tsc_capture_control_ptx is not supported
[   0.0830 ] ERROR: /misc/tsc_controls/tsc_capture_config_ptx is not supported
[   0.0831 ] ERROR: /misc/tsc_controls/tsc_stscrsr is not supported
[   0.0831 ] ERROR: /misc/tsc_controls/tsc_locking_adjust_num_control is not supported
[   0.0831 ] 
[   0.0831 ] Parsing config file :tegra234-mb1-bct-device-p3701-0000_cpp.dtb 
[   0.0831 ] Added Platform Config 9 data with size :- 100
[   0.0831 ] 
[   0.0831 ] Updating mb1-bct with firmware information
[   0.0837 ] tegrabct_v2 --chip 0x23 0 --mb1bct mb1_cold_boot_bct_MB1.bct --updatefwinfo readinfo_t234_min_prod.xml.bin
[   0.0851 ] tegrahost_v2 --chip 0x23 0 --align mb1_cold_boot_bct_MB1_aligned.bct
[   0.0862 ] tegrahost_v2 --chip 0x23 0 --magicid MBCT --appendsigheader mb1_cold_boot_bct_MB1_aligned.bct zerosbk
[   0.0868 ] adding BCH for mb1_cold_boot_bct_MB1_aligned.bct
[   0.0882 ] tegrasign_v3.py --key None --list mb1_cold_boot_bct_MB1_aligned_sigheader.bct_list.xml --pubkeyhash pub_key.key --sha sha512
[   0.0883 ] Assuming zero filled SBK key
[   0.0893 ] Warning: pub_key.key is not found
[   0.0891 ] tegrahost_v2 --chip 0x23 0 --updatesigheader mb1_cold_boot_bct_MB1_aligned_sigheader.bct.encrypt mb1_cold_boot_bct_MB1_aligned_sigheader.bct.hash zerosbk
[   0.0900 ] Generating recovery mb1-bct
[   0.0906 ] tegrabct_v2 --chip 0x23 0 --mb1bct mb1_bct.cfg --misc tegra234-mb1-bct-misc-p3701-0000_cpp.dtb --device tegra234-mb1-bct-device-p3701-0000_cpp.dtb
[   0.0911 ] MB1-BCT version: 0.10
[   0.0913 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_UNUSED5/ is not supported
[   0.0917 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_UNUSED5/ is not supported
[   0.0920 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_UNUSED5/ is not supported
[   0.0922 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_MCE_COVERAGE/ is not supported
[   0.0924 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_MCE_COVERAGE/ is not supported
[   0.0925 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_MCE_COVERAGE/ is not supported
[   0.0925 ] ERROR: /misc/tsc_controls/tsc_locking_config is not supported
[   0.0925 ] ERROR: /misc/tsc_controls/tsc_locking_diff_configuration is not supported
[   0.0925 ] ERROR: /misc/tsc_controls/tsc_locking_ref_frequency_configuration is not supported
[   0.0925 ] ERROR: /misc/tsc_controls/tsc_locking_control is not supported
[   0.0925 ] ERROR: /misc/tsc_controls/tsc_locking_adjust_configuration is not supported
[   0.0925 ] ERROR: /misc/tsc_controls/tsc_locking_fast_adjust_configuration is not supported
[   0.0925 ] ERROR: /misc/tsc_controls/tsc_locking_adjust_delta_control is not supported
[   0.0925 ] ERROR: /misc/tsc_controls/tsc_capture_control_ptx is not supported
[   0.0925 ] ERROR: /misc/tsc_controls/tsc_capture_config_ptx is not supported
[   0.0925 ] ERROR: /misc/tsc_controls/tsc_stscrsr is not supported
[   0.0925 ] ERROR: /misc/tsc_controls/tsc_locking_adjust_num_control is not supported
[   0.0925 ] 
[   0.0925 ] Parsing config file :tegra234-mb1-bct-device-p3701-0000_cpp.dtb 
[   0.0925 ] Added Platform Config 9 data with size :- 100
[   0.0925 ] 
[   0.0926 ] Updating mb1-bct with firmware information
[   0.0931 ] tegrabct_v2 --chip 0x23 0 --mb1bct mb1_bct_MB1.bct --recov --updatefwinfo readinfo_t234_min_prod.xml.bin
[   0.0946 ] tegrahost_v2 --chip 0x23 0 --align mb1_bct_MB1_aligned.bct
[   0.0956 ] tegrahost_v2 --chip 0x23 0 --magicid MBCT --appendsigheader mb1_bct_MB1_aligned.bct zerosbk
[   0.0961 ] adding BCH for mb1_bct_MB1_aligned.bct
[   0.0974 ] tegrasign_v3.py --key None --list mb1_bct_MB1_aligned_sigheader.bct_list.xml --pubkeyhash pub_key.key --sha sha512
[   0.0975 ] Assuming zero filled SBK key
[   0.0985 ] Warning: pub_key.key is not found
[   0.0983 ] tegrahost_v2 --chip 0x23 0 --updatesigheader mb1_bct_MB1_aligned_sigheader.bct.encrypt mb1_bct_MB1_aligned_sigheader.bct.hash zerosbk
[   0.0992 ] Error: Skip generating mem_bct because sdram_config is not defined
[   0.0992 ] Error: Skip generating mem_bct because sdram_config is not defined
[   0.0992 ] Copying signatures
[   0.0997 ] tegrahost_v2 --chip 0x23 0 --partitionlayout readinfo_t234_min_prod.xml.bin --updatesig images_list_signed.xml
[   0.1034 ] mb1_t234_prod_aligned_sigheader.bin.encrypt filename is from images_list
[   0.1035 ] psc_bl1_t234_prod_aligned_sigheader.bin.encrypt filename is from images_list
[   0.1035 ] Boot Rom communication
[   0.1040 ] tegrarcm_v2 --new_session --chip 0x23 0 --uid --download bct_br br_bct_BR.bct --download mb1 mb1_t234_prod_aligned_sigheader.bin.encrypt --download psc_bl1 psc_bl1_t234_prod_aligned_sigheader.bin.encrypt --download bct_mb1 mb1_bct_MB1_sigheader.bct.encrypt
[   0.1045 ] BR_CID: 0x80012344705DD5DD2C00000011FE8100
[   0.9204 ] Sending bct_br
[   1.7356 ] ERROR: might be timeout in USB write.
Error: Return value 3
Command tegrarcm_v2 --new_session --chip 0x23 0 --uid --download bct_br br_bct_BR.bct --download mb1 mb1_t234_prod_aligned_sigheader.bin.encrypt --download psc_bl1 psc_bl1_t234_prod_aligned_sigheader.bin.encrypt --download bct_mb1 mb1_bct_MB1_sigheader.bct.encrypt
Reading board information failed.
3

Hi,

the Reading board information failed. error sometimes means the EEPROM value of your board is somehow modified, and it won’t get fixed no matter how many times you try to re-flash it.
Are you using an official AGX Orin Devkit, or a custom carrier board? If you use a third-party carrier board, please contact the vendor first.

Also, can you elaborate more about

After reboot I can get press escape to enter a setup screen but I cannot boot.

Are you still at least able to log into your board via CLI?

4

Also, during recover-mode and flashing this ttyACM0 appears to be silent.

This means the port you are using is not the real serial console port.

The micor usb port is the true one. The type C port is just a usb port which cannot print early stage log.

5

Are you using an official AGX Orin Devkit, or a custom carrier board? If you use a third-party carrier board, please contact the vendor first.

This is the official AGX Orin Devkit, NOT a custom board.

Also, can you elaborate more about

After reboot I can get press escape to enter a setup screen but I cannot boot.

Just like the BIOS on a PC, if you watch the boot up of the ORIN you get an option to ‘ESC’ to a BIOS setup screen. However, proceeding to boot will ultimately fail. So the boot media just is nolonger valid.

Are you still at least able to log into your board via CLI?

Do you mean CLI via the USB TTY? or do you mean CLI via SSH? Either way, no. The TTY shows nothing.

6

Ok, I was wrong about this, during bootup I can see data on the serial USB console. It is silent when in recovery mode.

7

Could you share the log you see during boot so that we can make sure you are dumping correct log to go next step?

8

Hi Wayne,

Yes, it is very long though. Would you just like the error or would you like certain sections?

Here is the beginning

**  WARNING: Test Key is used.  **

L4TLauncher: Attempting Direct Boot
EFI stub: Booting Linux Kernel...
EFI stub: Using DTB from configuration table
EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path
EFI stub: Exiting boot services and installing virtual address map...
<FF><E4>I/TC: Secondary CPU 1 initializing
I/TC: Secondary CPU 1 switching to normal world boot
I/TC: Secondary CPU 2 initializing
I/TC: Secondary CPU 2 switching to normal world boot
I/TC: Secondary CPU 3 initializing
I/TC: Secondary CPU 3 switching to normal world boot
I/TC: Secondary CPU 4 initializing
I/TC: Secondary CPU 4 switching to normal world boot
I/TC: Secondary CPU 5 initializing
I/TC: Secondary CPU 5 switching to normal world boot
I/TC: Secondary CPU 6 initializing
I/TC: Secondary CPU 6 switching to normal world boot
I/TC: Secondary CPU 7 initializing
I/TC: Secondary CPU 7 switching to normal world boot
I/TC: Secondary CPU 8 initializing
I/TC: Secondary CPU 8 switching to normal world boot
I/TC: Secondary CPU 9 initializing
I/TC: Secondary CPU 9 switching to normal world boot
I/TC: Secondary CPU 10 initializing
I/TC: Secondary CPU 10 switching to normal world boot
I/TC: Secondary CPU 11 initializing
I/TC: Secondary CPU 11 switching to normal world boot
<FF><E1>[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd421]
[    0.000000] Linux version 5.10.104-tegra (buildbrain@mobile-u64-6289-d7000) (aarch64-buildroot-linux-gnu-gcc.br_real (Buildroot 2020.08) 9.3.0, GNU ld (GNU Binutils) 2.33.1) #1 SMP PREEMPT Sun Mar 19 07:55:28 PDT 2023
[    0.000000] OF: fdt: memory scan node memory@80000000, reg size 16,
[    0.000000] OF: fdt:  - 80000000 ,  c0000000
[    0.000000] Machine model: NVIDIA Orin Nano Developer Kit
[    0.000000] efi: EFI v2.70 by EDK II
[    0.000000] efi: RTPROP=0x808253f98 SMBIOS=0xffff0000 SMBIOS 3.0=0x806b60000 MEMATTR=0x80757a018 ESRT=0x806469118 RNG=0x82a9fdd18 MEMRESERVE=0x8075f8f18 
[    0.000000] efi: seeding entropy pool
[    0.000000] esrt: Reserving ESRT space from 0x0000000806469118 to 0x0000000806469150.
[    0.000000] Reserved memory: created CMA memory pool at 0x000000081a800000, size 256 MiB
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000080000000-0x00000000ffffffff]
[    0.000000]   DMA32    empty
[    0.000000]   Normal   [mem 0x0000000100000000-0x0000000837ffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080000000-0x00000000fffdffff]
[    0.000000]   node   0: [mem 0x00000000fffe0000-0x00000000ffffffff]
[    0.000000]   node   0: [mem 0x0000000100000000-0x0000000803ffffff]
[    0.000000]   node   0: [mem 0x0000000804000000-0x00000008061bffff]
[    0.000000]   node   0: [mem 0x00000008061c0000-0x00000008067fffff]
[    0.000000]   node   0: [mem 0x0000000806800000-0x000000080698bfff]
[    0.000000]   node   0: [mem 0x000000080698c000-0x0000000806b4ffff]
[    0.000000]   node   0: [mem 0x0000000806b50000-0x0000000806b6ffff]
[    0.000000]   node   0: [mem 0x0000000806b70000-0x00000008075cffff]
[    0.000000]   node   0: [mem 0x00000008075d0000-0x00000008075effff]
[    0.000000]   node   0: [mem 0x00000008075f0000-0x000000080760ffff]
[    0.000000]   node   0: [mem 0x0000000807610000-0x000000080762ffff]
[    0.000000]   node   0: [mem 0x0000000807630000-0x000000080769ffff]
[    0.000000]   node   0: [mem 0x00000008076a0000-0x000000080782ffff]
[    0.000000]   node   0: [mem 0x0000000807830000-0x000000080787ffff]
[    0.000000]   node   0: [mem 0x0000000807880000-0x00000008079bffff]
[    0.000000]   node   0: [mem 0x00000008079c0000-0x0000000807a4ffff]
[    0.000000]   node   0: [mem 0x0000000807a50000-0x0000000807ccffff]
[    0.000000]   node   0: [mem 0x0000000807cd0000-0x0000000807d0ffff]
[    0.000000]   node   0: [mem 0x0000000807d10000-0x0000000807d5ffff]

[    0.000000]   node   0: [mem 0x0000000807d60000-0x0000000807d8ffff]
[    0.000000]   node   0: [mem 0x0000000807d90000-0x0000000807d9ffff]
[    0.000000]   node   0: [mem 0x0000000807da0000-0x000000080837ffff]
[    0.000000]   node   0: [mem 0x0000000808380000-0x000000080841ffff]
[    0.000000]   node   0: [mem 0x0000000808420000-0x000000080851ffff]
[    0.000000]   node   0: [mem 0x0000000808520000-0x00000008085bffff]
[    0.000000]   node   0: [mem 0x00000008085c0000-0x000000080866ffff]
[    0.000000]   node   0: [mem 0x0000000808670000-0x000000080870ffff]
[    0.000000]   node   0: [mem 0x0000000808710000-0x000000080901ffff]
[    0.000000]   node   0: [mem 0x0000000809020000-0x00000008091fffff]
[    0.000000]   node   0: [mem 0x0000000809200000-0x000000082a9affff]
[    0.000000]   node   0: [mem 0x000000082a9b0000-0x000000082a9fffff]
[    0.000000]   node   0: [mem 0x000000082aa00000-0x000000082e7fffff]
[    0.000000]   node   0: [mem 0x000000082ea00000-0x000000082edfffff]
[    0.000000]   node   0: [mem 0x0000000836000000-0x0000000837ffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x0000000837ffffff]
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.1 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: Trusted OS migration not required
[    0.000000] psci: SMC Calling Convention v1.2
[    0.000000] percpu: Embedded 32 pages/cpu s90200 r8192 d32680 u131072
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: detected: GIC system register CPU interface
[    0.000000] CPU features: detected: Virtualization Host Extensions
[    0.000000] CPU features: kernel page table isolation forced ON by KASLR
[    0.000000] CPU features: detected: Kernel page table isolation (KPTI)
[    0.000000] CPU features: detected: Spectre-v4
[    0.000000] alternatives: patching kernel code
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 7937536
[    0.000000] Kernel command line: root=/dev/mmcblk1p1 rw rootwait rootfstype=ext4 mminit_loglevel=4 console=ttyTCU0,115200 console=ttyAMA0,115200 firmware_class.path=/etc/firmware fbcon=map:0 net.ifnames=0

and here is the error, this error will just repeat:

[   11.413243] CPU:0, Error: cbb-fabric@0x13a00000, irq=25
[   11.418613] **************************************
[   11.423535] CPU:0, Error:cbb-fabric, Errmon:2
[   11.428019]    Error Code            : TIMEOUT_ERR
[   11.432060] 
[   11.433585]    Error Code            : TIMEOUT_ERR
[   11.437613]    MASTER_ID             : CCPLEX
[   11.441108]    Address               : 0x3f40078
[   11.444693]    Cache                 : 0x1 -- Bufferable 
[   11.448983]    Protection            : 0x2 -- Unprivileged, Non-Secure, Data Access
[   11.455962]    Access_Type           : Read
[   11.459448]    Access_ID             : 0x16
[   11.459449]    Fabric                : cbb-fabric
[   11.466334]    Slave_Id              : 0x33
[   11.469549]    Burst_length          : 0x0
[   11.473036]    Burst_type            : 0x1
[   11.476347]    Beat_size             : 0x2
[   11.479563]    VQC                   : 0x0
[   11.482336]    GRPSEC                : 0x7e
[   11.485382]    FALCONSEC             : 0x0
[   11.488609]  **************************************
[   11.493632] ------------[ cut here ]------------
[   11.498386] WARNING: CPU: 0 PID: 170 at drivers/soc/tegra/cbb/tegra234-cbb.c:577 tegra234_cbb_isr+0x130/0x170
[   11.508565] Modules linked in:
[   11.511700] CPU: 0 PID: 170 Comm: kworker/0:3 Not tainted 5.10.104-tegra #1
[   11.518852] Hardware name: Unknown Jetson AGX Orin/Jetson AGX Orin, BIOS 3.1-32827747 03/19/2023
[   11.527881] Workqueue: events deferred_probe_work_func
[   11.533153] pstate: 60400089 (nZCv daIf +PAN -UAO -TCO BTYPE=--)
[   11.539306] pc : tegra234_cbb_isr+0x130/0x170
[   11.543772] lr : tegra234_cbb_isr+0x10c/0x170
[   11.548238] sp : ffff800010003b80
[   11.551638] x29: ffff800010003b80 x28: ffff3ecc0485ab80 
[   11.557088] x27: 0000000000000001 x26: 0000000000000080 
[   11.562538] x25: ffffa0e111b09ed0 x24: ffffa0e11245be40 
[   11.567981] x23: ffffa0e111df7000 x22: 0000000000000019 
[   11.573434] x21: ffffa0e11227ef20 x20: 0000000000000002 
[   11.578882] x19: ffffa0e11227ef10 x18: 0000000000000010 
[   11.584331] x17: 0000000000000000 x16: 0000000000000000 
[   11.589794] x15: ffff3ecc0485b0f0 x14: ffffffffffffffff 
[   11.595256] x13: ffff800090003687 x12: ffff80001000368f 
[   11.600720] x11: 0101010101010101 x10: 7f7f7f7f7f7f7f7f 
[   11.606182] x9 : ffff8000100039a0 x8 : 2a2a2a2a2a2a2a2a 
[   11.611638] x7 : 2a2a2a2a2a2a2a09 x6 : c0000000ffffefff 
[   11.617102] x5 : 0000000000057fa8 x4 : ffffa0e112107968 
[   11.622558] x3 : 0000000000000000 x2 : ffffa0e11058e170 
[   11.628019] x1 : ffff3ecc0485ab80 x0 : 0000000100010100 
[   11.633477] Call trace:
[   11.635987]  tegra234_cbb_isr+0x130/0x170
[   11.640106]  __handle_irq_event_percpu+0x68/0x2a0
[   11.644939]  handle_irq_event_percpu+0x40/0xa0
[   11.649498]  handle_irq_event+0x50/0xf0
[   11.653433]  handle_fasteoi_irq+0xc0/0x170
[   11.657646]  generic_handle_irq+0x40/0x60
[   11.661772]  __handle_domain_irq+0x70/0xd0
[   11.665983]  gic_handle_irq+0x68/0x134
[   11.669833]  el1_irq+0xd0/0x180
[   11.673052]  __do_softirq+0xb4/0x3e8
[   11.676727]  irq_exit+0xc0/0xe0
[   11.679942]  __handle_domain_irq+0x74/0xd0
[   11.684148]  gic_handle_irq+0x68/0x134
[   11.688000]  el1_irq+0xd0/0x180
[   11.691218]  tegra_p2u_power_on+0x3c/0x150
[   11.695431]  phy_power_on+0x98/0x110
[   11.699106]  tegra_pcie_enable_phy+0x68/0xf0
[   11.703495]  tegra_pcie_config_controller+0x138/0x450
[   11.708685]  tegra_pcie_config_rp+0x4c/0x250
[   11.713076]  tegra_pcie_dw_probe+0xae8/0x1150
[   11.717549]  platform_drv_probe+0x5c/0xb0
[   11.721667]  really_probe+0xf8/0x3d0
[   11.725336]  driver_probe_device+0x60/0xc0
[   11.729548]  __device_attach_driver+0x8c/0xd0
[   11.734027]  bus_for_each_drv+0x8c/0xe0
[   11.737975]  __device_attach+0xf8/0x160
[   11.741912]  device_initial_probe+0x28/0x40
[   11.746203]  bus_probe_device+0xa4/0xb0
[   11.750140]  deferred_probe_work_func+0x90/0xd0
[   11.754797]  process_one_work+0x1c4/0x4a0
[   11.758913]  worker_thread+0x200/0x430
[   11.762765]  kthread+0x148/0x170
[   11.766073]  ret_from_fork+0x10/0x24
[   11.769742] ---[ end trace 59fa5dc55e1de70b ]---
[   11.774534] CPU:0, Error: cbb-fabric@0x13a00000, irq=25
[   11.779897] **************************************
[

Thank you.

9

Hi CapnSaicin,

Is there any log prior to the log you dumped? If it does not have, then you are still using the wrong port to dump log.

I already told it in previous comment. I need "micro usb port’. Not “type C”. Do you need me to share the photo of the difference of these two usb ports?

10

If you are sure about the port you are using is micro usb port, could you take a photo of the board you are using?

11

Hi WayneWWW,

Yes, this is absolutely the micro USB port and not the USB C. Neither of the USB-C are being used at the moment. Yes, there is more prior to this, I had assumed you were interested in the Linux boot up.

I have attached everything from power up.

minicom4.cap (124.4 KB)

12

Hi CapnSaicin,

It looks like the log is correct.

But could you really share the photo of your device? I am confused. The log says you are using a Orin NX/Nano (or at least its image) but not a Orin AGX 64GB.

Of course that could be the reason that boot fails. But I don’t know why that image gets in use on your board. What did you do to this board before filing this topic?

Also, are you trying to boot from a sdcard?

13

Back to the flash process,

What we need here is the flash log from the serial console port and host side. You need to open the serial console, start the flash process and then see if anything got printed on serial console.

We need both side log (especially the serial console part) to tell why the flash goes failure.

Even a “no log got printed from serial console” could be info to us. So please share.

14

This is an AGX Orin Dev Kit. Note I am pointing to the micro USB port.

20230504_124202
20230504_1242021920×1440 259 KB

I have booted this device from SD but this boot log is without SD. When first trying to debug the failure, I downloaded and booted the Orin Nano SD card image. But that SD-card is not in the system at this time. That SD image does work to boot into linux, but for some reason I cannot get the card to boot from EMMC.

When I first received the system the EMMC boot was fine but I wanted the latest jetpack and so I ran upgrade-dist after which it would nolonger boot. This lead me down the path to finally trying the SD Card image. However, the above info about the SDK Manager failing all pre-date booting with the SD card.

Thank you

15

I will try to capture what you are asking for and report back.

16

When in recovery mode and trying to flash I see nothing happen on the ttyACM0 (from micro USB). The host side output from flash.sh can be seen above in my previous post, nothing has changed there.

17

Then it is usb detection issue on host. Try to enable these logs on your host and start the flash process. Share the dmesg on your host.

echo 8 > /proc/sys/kernel/printk
echo ‘module usbcore +p’ > /sys/kernel/debug/dynamic_debug/control
echo ‘module xhci_hcd +p’ > /sys/kernel/debug/dynamic_debug/control

You could also try other usb cable/ports (host side) or other host machines.

18

I have seen this recommended in another post. Can you provide the exact steps, when do I apply this? Do I first put the unit in recovery mode, then apply the above echo commands and then run SDK manager? If you will provide more guidance then I will give this a try tomorrow.

19

You can add that before you put the board into recovery mode and then run sdkmanager.

20

Hi Wayne,

attached is the dmesg after performing the prescribed commands and trying to flash via SDK manager.

dmesg.txt (174.3 KB)

Thank you.

21

Is there any log before this? Actually you one shared a 4 second log.