Unable to Flash NVME Jetson AGX Orin 64GB Dev kit with supplied USB Cable

I have been attempting to flash the Jetson AGX Orin 64 GB Developer kit across many virtual and physical hardware configurations as host running the SDK manager on Ubuntu 20.04.6 LTS

At this point I know that I must be on physical hardware, not virtual machine to do this. I’m on a Dell XPS 13 9360. I have about 15 hours of hands on at this point trying to do this.

I have tested across all 3 USB ports, including the USB C port directly with a high end cable that supports data/power.

I have updated 20.04 with software/system updates, the ubunu version (as stated) is Ubuntu 20.04.6

I keep ending up at the point where the SDK manager displayes the error window
“The connected Jetson device is not ready for the flash”

My NVME drive has no partitions on it (cleared as indicated from other articles I’ve read here).

I have attempted both automatic and manual runtime based install

When checking from the host with lsusb I see:

agrajag@tor-XPS:~/Desktop$ lsusb
Bus 004 Device 002: ID 0955:7020 NVIDIA Corp. L4T (Linux for Tegra) running on Tegra
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 002: ID 0451:8440 Texas Instruments, Inc.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 007: ID 0bda:568b Realtek Semiconductor Corp. Integrated_Webcam_HD
Bus 001 Device 005: ID 04f3:20d0 Elan Microelectronics Corp. Touchscreen
Bus 001 Device 003: ID 0cf3:e301 Qualcomm Atheros Communications
Bus 001 Device 008: ID 0451:82ff Texas Instruments, Inc. Integrated_Webcam_HD
Bus 001 Device 006: ID 1532:0067 Razer USA, Ltd
Bus 001 Device 004: ID 1532:026c Razer USA, Ltd
Bus 001 Device 002: ID 0451:8442 Texas Instruments, Inc.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

UFW is disabled

how do I troubleshoot the flashing process further?

I have attempted all these reccomended steps, repeatedly with no change in behavior, it alwats fails with the same as showin in the attached image to this note.

So I’ve iterated through stripping down the installation to ONLY do the flash of the Jetson AGX Orin unit. It errors out, and then I’m iterating across multiple attempts following what the Warning Error Dialog that I’ve shown is stating to do; switch USB port, change out cable (I’m using the one included with the unit…), use hub/dont use hub, power cycling and setting the unit based on the instructions to be ready for firmware flash.

When I export the logs after doing that and going through them, I’m not finding a log that would represent the flash attempt, so I’m not finding anything to troubleshoot over?

Is there another log in any location that is not pulled into the zip file or can I attach monitor/keyboard to the unit I’m attempting to flash and locate anything there indicating the error? It is appearing that if I skip the flashing portion of this, no log file is produced?

Initially I was having apt errors due to gpg keys missing, but I’ve corrected that, removed an unrelated repo that was throwing errors (grafana). I even followed your network repo install steps for the SDK manager to make sure I’ve imported your GPG keys to avoid any further erors. I have verified sudo apt update runs on the command line without errors. I’m having no luck determining where the flash process is failing. I have allowed the install to complete the host SDK setup and that worked, but everything fails point of attempting flashing the Jetson Orin Nano unit.

Thanks in advance for your help.

Going through the default_replay.txt this command appeared to be significant, and this is what I’m seeing returned, some kind of error towards the end, indicating USB timeout?

agrajag@tor-XPS:/tmp$ sudo /home/agrajag/nvidia/nvidia_sdk/JetPack_6.2_Linux_JETSON_AGX_ORIN_TARGETS/Linux_for_Tegra/nvautoflash.sh --print_boardid --usb-instance 3-1.2.2 ; echo $?
[sudo] password for agrajag:
*** Checking ONLINE mode … OK.
*** Checking target board connection … 1 connections found.
*** Reading ECID … FUSELEVEL=fuselevel_production hwchipid=0x23 bootauth=NS
*** Reading EEPROM … “/home/agrajag/nvidia/nvidia_sdk/JetPack_6.2_Linux_JETSON_AGX_ORIN_TARGETS/Linux_for_Tegra/bootloader/tegraflash.py” --chip 0x23 --instance 3-1.2.2 --applet “/home/agrajag/nvidia/nvidia_sdk/JetPack_6.2_Linux_JETSON_AGX_ORIN_TARGETS/Linux_for_Tegra/bootloader/mb1_t234_prod.bin” --cfg readinfo_t234_min_prod.xml --dev_params tegra234-br-bct-diag-boot.dts --device_config tegra234-mb1-bct-device-p3701-0000.dts --misc_config tegra234-mb1-bct-misc-p3701-0000.dts --bins “mb2_applet applet_t234.bin” --skipuid --cmd “dump eeprom cvm cvm.bin; reboot recovery”
Welcome to Tegra Flash
version 1.0.0
Type ? or help for help and q or quit to exit
Use ! to execute system commands

[ 0.0377 ] tegrarcm_v2 --instance 3-1.2.2 --chip 0x23 0 --ismb2applet
[ 0.0391 ] File rcm_state open failed
[ 0.0396 ] ERROR: failed to read rcm_state
[ 0.0396 ]
[ 0.0410 ] tegrasign_v3.py --key None --getmode mode.txt
[ 0.0415 ] Assuming zero filled SBK key
[ 0.0405 ] Pre-processing config: tegra234-mb1-bct-device-p3701-0000.dts
[ 0.0513 ] Pre-processing config: tegra234-mb1-bct-misc-p3701-0000.dts
[ 0.0750 ] Parsing partition layout
[ 0.0776 ] tegraparser_v2 --pt readinfo_t234_min_prod.xml.tmp
[ 0.0808 ] Kernel DTB used: None
[ 0.0808 ] WARNING: dce base dtb is not provided

[ 0.0809 ] Parsing partition layout
[ 0.0826 ] tegraparser_v2 --pt readinfo_t234_min_prod.xml.tmp
[ 0.0846 ] Creating list of images to be signed
[ 0.0863 ] tegrahost_v2 --chip 0x23 0 --partitionlayout readinfo_t234_min_prod.xml.bin --list images_list.xml zerosbk
[ 0.0879 ] MB1: Nvheader already present is mb1_t234_prod_aligned.bin
[ 0.0913 ] Header already present for mb1_t234_prod_aligned_sigheader.bin
[ 0.0945 ] MB1: Nvheader already present is mb1_t234_prod_aligned.bin
[ 0.1048 ] Header already present for mb1_t234_prod_aligned_sigheader.bin
[ 0.1088 ] MB1: Nvheader already present is psc_bl1_t234_prod_aligned.bin
[ 0.1175 ] Header already present for psc_bl1_t234_prod_aligned_sigheader.bin
[ 0.1184 ] adding BCH for mb2_t234_aligned.bin
[ 0.1239 ] MB1: Nvheader already present is psc_bl1_t234_prod_aligned.bin
[ 0.1459 ] Header already present for psc_bl1_t234_prod_aligned_sigheader.bin
[ 0.1468 ] adding BCH for mb2_t234_aligned.bin
[ 0.1737 ] Filling MB1 storage info
[ 0.1738 ] Parsing dev params for multi chains
[ 0.1842 ] Generating br-bct
[ 0.1858 ] Updating dev and MSS params in BR BCT
[ 0.1859 ] tegrabct_v2 --dev_param tegra234-br-bct-diag-boot_cpp.dtb --brbct br_bct.cfg --chip 0x23 0
[ 0.1877 ] Updating bl info
[ 0.1902 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --updateblinfo readinfo_t234_min_prod.xml.bin
[ 0.1926 ] WARNING: boot chain is not completed. set to 0
[ 0.1948 ] Generating signatures
[ 0.1961 ] tegrasign_v3.py --key None --list images_list.xml --pubkeyhash pub_key.key --sha sha512
[ 0.1967 ] Assuming zero filled SBK key
[ 0.2267 ] Warning: pub_key.key is not found
[ 0.2260 ] Parsing dev params for multi chains
[ 0.2261 ] Generating br-bct
[ 0.2281 ] Updating dev and MSS params in BR BCT
[ 0.2283 ] tegrabct_v2 --dev_param tegra234-br-bct-diag-boot_cpp.dtb --brbct br_bct.cfg --chip 0x23 0
[ 0.2310 ] Updating bl info
[ 0.2336 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --updateblinfo readinfo_t234_min_prod.xml.bin --updatesig images_list_signed.xml
[ 0.2354 ] WARNING: boot chain is not completed. set to 0
[ 0.2369 ] Generating SHA2 Hash
[ 0.2407 ] Sha saved in br_bct_BR.sha
[ 0.2397 ] Get Signed section of bct
[ 0.2411 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --listbct bct_list.xml
[ 0.2428 ] Signing BCT
[ 0.2442 ] tegrasign_v3.py --key None --list bct_list.xml --pubkeyhash pub_key.key --sha sha512
[ 0.2446 ] Assuming zero filled SBK key
[ 0.2495 ] Sha saved in br_bct_BR.sha
[ 0.2502 ] Warning: pub_key.key is not found
[ 0.2490 ] Updating BCT with signature
[ 0.2508 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --updatesig bct_list_signed.xml
[ 0.2524 ] Offset :4608 Len :3584
[ 0.2533 ] Generating SHA2 Hash
[ 0.2546 ] tegrasign_v3.py --key None --list bct_list.xml --sha sha512
[ 0.2551 ] Assuming zero filled SBK key
[ 0.2552 ] Assuming zero filled SBK key
[ 0.2638 ] Sha saved in br_bct_BR.sha
[ 0.2635 ] Updating BCT with SHA2 Hash
[ 0.2653 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --updatesha bct_list_signed.xml
[ 0.2670 ] Offset :4608 Len :3584
[ 0.2677 ] Offset :68 Len :8124
[ 0.2677 ]
[ 0.2680 ] Generating coldboot mb1-bct
[ 0.2697 ] tegrabct_v2 --chip 0x23 0 --mb1bct mb1_cold_boot_bct.cfg --misc tegra234-mb1-bct-misc-p3701-0000_cpp.dtb --device tegra234-mb1-bct-device-p3701-0000_cpp.dtb
[ 0.2712 ] MB1-BCT version: 0.13

[ 0.2740 ] Parsing config file :tegra234-mb1-bct-device-p3701-0000_cpp.dtb
[ 0.2746 ] Added Platform Config 9 data with size :- 100
[ 0.2746 ]
[ 0.2747 ] Updating mb1-bct with firmware information
[ 0.2764 ] tegrabct_v2 --chip 0x23 0 --mb1bct mb1_cold_boot_bct_MB1.bct --updatefwinfo readinfo_t234_min_prod.xml.bin
[ 0.2801 ] tegrahost_v2 --chip 0x23 0 --align mb1_cold_boot_bct_MB1_aligned.bct
[ 0.2819 ] Generating SHA2 Hash for mb1bct
[ 0.2865 ] Sha saved in mb1_cold_boot_bct_MB1_aligned.sha
[ 0.2902 ] Sha saved in mb1_cold_boot_bct_MB1.sha
[ 0.2921 ] tegrahost_v2 --chip 0x23 0 --magicid MBCT --appendsigheader mb1_cold_boot_bct_MB1_aligned.bct zerosbk
[ 0.2944 ] adding BCH for mb1_cold_boot_bct_MB1_aligned.bct
[ 0.2975 ] tegrasign_v3.py --key None --list mb1_cold_boot_bct_MB1_aligned_sigheader.bct_list.xml --pubkeyhash pub_key.key --sha sha512
[ 0.2978 ] Assuming zero filled SBK key
[ 0.3012 ] Warning: pub_key.key is not found
[ 0.3019 ] tegrahost_v2 --chip 0x23 0 --updatesigheader mb1_cold_boot_bct_MB1_aligned_sigheader.bct.encrypt mb1_cold_boot_bct_MB1_aligned_sigheader.bct.hash zerosbk
[ 0.3046 ] Generating recovery mb1-bct
[ 0.3062 ] tegrabct_v2 --chip 0x23 0 --mb1bct mb1_bct.cfg --misc tegra234-mb1-bct-misc-p3701-0000_cpp.dtb --device tegra234-mb1-bct-device-p3701-0000_cpp.dtb
[ 0.3078 ] MB1-BCT version: 0.13

[ 0.3105 ] Parsing config file :tegra234-mb1-bct-device-p3701-0000_cpp.dtb
[ 0.3110 ] Added Platform Config 9 data with size :- 100
[ 0.3110 ]
[ 0.3111 ] Updating mb1-bct with firmware information
[ 0.3137 ] tegrabct_v2 --chip 0x23 0 --mb1bct mb1_bct_MB1.bct --recov --updatefwinfo readinfo_t234_min_prod.xml.bin
[ 0.3193 ] tegrahost_v2 --chip 0x23 0 --align mb1_bct_MB1_aligned.bct
[ 0.3221 ] Generating SHA2 Hash for mb1bct
[ 0.3278 ] Sha saved in mb1_bct_MB1_aligned.sha
[ 0.3306 ] Sha saved in mb1_bct_MB1.sha
[ 0.3310 ] tegrahost_v2 --chip 0x23 0 --magicid MBCT --appendsigheader mb1_bct_MB1_aligned.bct zerosbk
[ 0.3325 ] adding BCH for mb1_bct_MB1_aligned.bct
[ 0.3356 ] tegrasign_v3.py --key None --list mb1_bct_MB1_aligned_sigheader.bct_list.xml --pubkeyhash pub_key.key --sha sha512
[ 0.3360 ] Assuming zero filled SBK key
[ 0.3395 ] Warning: pub_key.key is not found
[ 0.3403 ] tegrahost_v2 --chip 0x23 0 --updatesigheader mb1_bct_MB1_aligned_sigheader.bct.encrypt mb1_bct_MB1_aligned_sigheader.bct.hash zerosbk
[ 0.3432 ] Info: Skip generating mem_bct because sdram_config is not defined
[ 0.3433 ] Info: Skip generating mem_bct because sdram_config is not defined
[ 0.3433 ] Copying signatures
[ 0.3457 ] tegrahost_v2 --chip 0x23 0 --partitionlayout readinfo_t234_min_prod.xml.bin --updatesig images_list_signed.xml
[ 0.3605 ] mb1_t234_prod_aligned_sigheader.bin.encrypt filename is from images_list
[ 0.3608 ] psc_bl1_t234_prod_aligned_sigheader.bin.encrypt filename is from images_list
[ 0.3608 ] Boot Rom communication
[ 0.3625 ] tegrarcm_v2 --instance 3-1.2.2 --new_session --chip 0x23 0 --uid --download bct_br br_bct_BR.bct --download mb1 mb1_t234_prod_aligned_sigheader.bin.encrypt --download psc_bl1 psc_bl1_t234_prod_aligned_sigheader.bin.encrypt --download bct_mb1 mb1_bct_MB1_sigheader.bct.encrypt
[ 0.3639 ] BR_CID: 0x80012344705DF2142000000018010180
[ 0.3719 ] Sending bct_br
[ 0.3786 ] ERROR: might be timeout in USB write.
Error: Return value 3
Command tegrarcm_v2 --instance 3-1.2.2 --new_session --chip 0x23 0 --uid --download bct_br br_bct_BR.bct --download mb1 mb1_t234_prod_aligned_sigheader.bin.encrypt --download psc_bl1 psc_bl1_t234_prod_aligned_sigheader.bin.encrypt --download bct_mb1 mb1_bct_MB1_sigheader.bct.encrypt
— Error: Reading board information failed.

Hi,

Do you put your device in the recovery mode?

Please refer to the Quick Start — NVIDIA Jetson Linux Developer Guide 1 documentation to check whether the device in recovery mode.

Thanks

Yes, I am putting it into recovery mode each iteration of the attempts. Turn unit off press middle recovery button and hold. push and hold power button for a moment, continuing holding recovery, release recovery.

I am verifying with lsusb and see the appropriate response for it. My first post has the output and screenshot is showing that in the background.

If you see the error happened right after “BR_CID” and “sending bct_br” in the log, then that is an error caused by the usb suspend function on the host.

Please refer to

Thank you very much both of you, I will work through that and verify my versions and patching and update here.

1 Like

Ok so I got past the point of failure applying the FAQ’s instruction of

Q: I get a USB timeout error during flash Orin. How to resolve that?

A known issue for ubuntu 20.04 host is the USB autosuspend function is enabled on it by default. This leads to Jetson recovery mode gets suspended before flash. To bypass this error, need to:

  1. Disable autosuspend on host PC with below command:

sudo -s
echo -1 > /sys/module/usbcore/parameters/autosuspend

  1. Plug out the USB cable, put jetson back into recovery mode again and plug the usb cable back.

I’m reviewing the apply patch instructions as well for PCN210100 & One thing that is very confusing is the version strings, I’m applying jetpack 6.2, so I’m assuming those patches are still required? Those patches are to be applied to the host (physical ubuntu 20.04 system?) The version that is being installed with jetpack 6.2 is NVIDIA Jetson Linux 36.4.3, thus my confusion (the release page for it has no overlay patches listed Jetson Linux | NVIDIA Developer )

I’m using the SDK Manager 2.2.0 12028, not the command line scripts. My configuration goal is to flash/install to the NVME i’ve installed. I’m assuming (right now, its been running for 20 min or so) that the disk format and then load of the Jetson Linux will take a while?

So to summarize:

Patches not applied, not understanding WHERE they should be applied, on the host linux system, what path?

Will the SDK Manager GUI pick up on the patching or do I need to switch to performing this process of flashing / installing over NVME SSD via command line? If so is there a walkthrough? I’m getting lost in the cross referencing documentation.

I’m assuming the NVME SSD should NOT be formatted ahead of this flashing procedure (other articles indicated that it should not be pre-formatted).

I appear to be hanging at “installing 31.72%”, flashing is indicating 0%

  • is there a path I can be checking on the host system for logging that will help me determine current state of what the SDK Manager is doing? (I was having it export logs before).

Thanks in advance

Ended up killing the NVSDK process at 1 hour,
Powered off the unit
Got this as far as logging ~/.nvsdkm/logs/JetPack_6.2_Linux/NV_L4T_FLASH_JETSON_LINUX_COMP.log
NV_L4T_FLASH_JETSON_LINUX_COMP.log (16.3 KB)

ufw is not running

agrajag@tor-XPS:~/.nvsdkm/logs/JetPack_6.2_Linux$ sudo ufw status
[sudo] password for agrajag:

From end of that uploaded log:

16:01:36.394 - Info: Please install the Secureboot package to use initrd flash for fused board
16:01:36.414 - Info: # Entry added by NVIDIA initrd flash tool
16:01:36.426 - Info: /home/agrajag/nvidia/nvidia_sdk/JetPack_6.2_Linux_JETSON_AGX_ORIN_TARGETS/Linux_for_Tegra/tools/kernel_flash/tmp 127.0.0.1(rw,nohide,insecure,no_subtree_check,async,no_root_squash)
16:01:36.430 - Error: rpcbind: another rpcbind is already running. Aborting
16:01:36.707 - Error: Job for nfs-server.service canceled.
16:01:36.789 - Error: [exec_command]: /bin/bash -c /home/agrajag/.nvsdkm/replays/scripts/JetPack_6.2_Linux/NV_L4T_FLASH_JETSON_LINUX_COMP.sh; [error]: Job for nfs-server.service canceled.
16:01:36.789 - Info: [ Component Install Finished with Error ]16:01:36.790 - Info: [host] [ 1.94 MB used. Disk Avail on Partition /home/agrajag/.Private: 328.12 GB ]16:01:36.790 - Info: [ NV_L4T_FLASH_JETSON_LINUX_COMP Install took 4s ]
Status: inactive

checking there is a running rpcbind, there is and you have to completely disable it per discussion here;

systemctl disable rpcbind.target
systemctl disable rpcbind.socket
systemctl disable rpcbind.service
systemctl stop rpcbind.target
systemctl stop rpcbind.socket
systemctl stop rpcbind.service

At that point restarting the SDK Manager after preparing the Jetson AGX Orion for recovery mode started working and I now see the “flashing” and “installing” progress bar’s from the UI showing progression…
(picture of screen from above when it was failing)

I’ll update if it was completely successful.

So it got to 99% flash (after I walked away for a while), when I came back there was a warning dialog stating that it was taking longer than expected, I dismissed it and saw the attached state of 99% completed at flashing. install at 37.44%. Attached are the uart serial port and logs from sdk manager, as well as the process (ps -ef) output which has a set of SDK manager processes one with a huge amount of information and command that must be what was hung?
ps-out.txt (28.5 KB)

I’ll post an additional note on where the unit is when I attempt to reboot it, later. Please let me know what is indicated took place tho from your understanding of these logs.


NV_L4T_FLASH_JETSON_LINUX_COMP.log (15.8 KB)
uart-serial-jetson-agx-orin-flash-session.txt (5.0 KB)

yeah from the looks of things on unit, it did nothing, but faked the UI in the host SDK manager that it was progressing, please check out the attached files and advise.

Maybe it is just me, but ubuntu 20.04 failed as host for sdkmanager. It has consistently worked from an ubuntu 22.04 host.

I also ran this on ubuntu 22.04 host to install flash dependencies:
sudo ./Linux_for_Tegra/tools/l4t_flash_prerequisites.sh

Perhaps the release notes or documentation should be reflecting this, thanks for the tip.

If you are using latest Jetpack6.2, then it already include the PCN patch. You don’t need to apply it again.

And actually we don’t provide a PCN overlay file for Jetpack6.2 at all. I really don’t know what you want to apply here.

What is the current status of this issue?

I looked into the last log you shared but looks like the flash process did not even start yet.

18:02:18.023 - Info: Please install the Secureboot package to use initrd flash for fused board

Is this board got fused before? Not a fresh module?

New unit purchased in January, installed nvme ssd goal is to get using SSD to be root disk for boot of unit.

Unable to get SDKManager to properly flash unit and install Jetson os on nvme as root disk

I am following what was stated in previous forum updates with regard to patching, but it looks like that was not a valid thing to recommend…

If there is nothing of value for troubleshooting in the logs and console output provided then it appears that 20.04 is not an option for running current release of SDK Manager

Just to clarify. And please be aware that what I am talking about is just a debug attempt.

Is sdkdmanger able to flash this Orin AGX eMMC instead of NVMe? Did you ever try that ?

I didn’t mean you just use eMMC forever. This is just a test.

I am following what was stated in previous forum updates with regard to patching, but it looks like that was not a valid thing to recommend…

It is not “I don’t recommend”. It is you totally don’t need to do that on Jetpack6.2. Jetpack6.2 already included that patch. You don’t need to apply patch to something that already has that patch.