Sdkmanager flash error

Autonomous Machines Jetson & Embedded Systems Jetson AGX Orin
, ,
1

When I was using sdkmanager to flash Jetson Orin Devkit I ran into “failed to read rcm_state” error. I tried both 1.8.4 and 1.9.1 they have the same error. I followed the instruction and set the board to force recovery mode before flash. Can anyone help what could be the problem? Thank.

=====
[   0.0184 ] tegrarcm_v2 --chip 0x23 0 --ismb2applet
[   0.0193 ] File rcm_state open failed
[   0.0197 ] **ERROR: failed to read rcm_state**
[   0.0203 ] Pre-processing config: tegra234-mb1-bct-device-p3701-0000.dts
[   0.0303 ] Pre-processing config: tegra234-mb1-bct-misc-p3701-0000.dts
[   0.0398 ] Parsing partition layout
[   0.0435 ] tegraparser_v2 --pt readinfo_t234_min_prod.xml.tmp
[   0.0451 ] Parsing partition layout
[   0.0458 ] tegraparser_v2 --pt readinfo_t234_min_prod.xml.tmp
[   0.0467 ] Creating list of images to be signed
[   0.0496 ] tegrahost_v2 --chip 0x23 0 --partitionlayout readinfo_t234_min_prod.xml.bin --list images_list.xml zerosbk
[   0.0502 ] MB1: Nvheader already present is mb1_t234_prod_aligned.bin
[   0.0518 ] Header already present for mb1_t234_prod_aligned_sigheader.bin
[   0.0520 ] MB1: Nvheader already present is mb1_t234_prod_aligned.bin
[   0.0557 ] Header already present for mb1_t234_prod_aligned_sigheader.bin
[   0.0565 ] MB1: Nvheader already present is psc_bl1_t234_prod_aligned.bin
[   0.0620 ] Header already present for psc_bl1_t234_prod_aligned_sigheader.bin
[   0.0623 ] adding BCH for mb2_t234_aligned.bin
[   0.0658 ] MB1: Nvheader already present is psc_bl1_t234_prod_aligned.bin
[   0.0776 ] Header already present for psc_bl1_t234_prod_aligned_sigheader.bin
[   0.0778 ] adding BCH for mb2_t234_aligned.bin
[   0.0912 ] Generating signatures
[   0.0929 ] tegrasign_v3.py --key None --list images_list.xml --pubkeyhash pub_key.key --sha sha512
[   0.0930 ] Assuming zero filled SBK key
[   0.1306 ] Warning: pub_key.key is not found
[   0.1297 ] Parsing dev params for multi chains
[   0.1460 ] Generating br-bct
[   0.1521 ] Updating dev and MSS params in BR BCT
[   0.1523 ] tegrabct_v2 --dev_param tegra234-br-bct-diag-boot_cpp.dtb --brbct br_bct.cfg --chip 0x23 0
[   0.1551 ] Updating bl info
[   0.1580 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --updateblinfo readinfo_t234_min_prod.xml.bin --updatesig images_list_signed.xml
[   0.1605 ] WARNING: boot chain is not completed. set to 0
[   0.1628 ] Get Signed section of bct
[   0.1652 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --listbct bct_list.xml
[   0.1677 ] Signing BCT
[   0.1695 ] tegrasign_v3.py --key None --list bct_list.xml --pubkeyhash pub_key.key --sha sha512
[   0.1699 ] Assuming zero filled SBK key
[   0.1775 ] Sha saved in br_bct_BR.sha
[   0.1784 ] Warning: pub_key.key is not found
[   0.1769 ] Updating BCT with signature
[   0.1794 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --updatesig bct_list_signed.xml
[   0.1817 ] Offset :4608 Len :3584
[   0.1827 ] Generating SHA2 Hash
[   0.1844 ] tegrasign_v3.py --key None --list bct_list.xml --sha sha512
[   0.1849 ] Assuming zero filled SBK key
[   0.1849 ] Assuming zero filled SBK key
[   0.1922 ] Sha saved in br_bct_BR.sha
[   0.1920 ] Updating BCT with SHA2 Hash
[   0.1938 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --updatesha bct_list_signed.xml
[   0.1954 ] Offset :4608 Len :3584
[   0.1959 ] Offset :68 Len :8124
[   0.1962 ] Generating coldboot mb1-bct
[   0.1976 ] tegrabct_v2 --chip 0x23 0 --mb1bct mb1_cold_boot_bct.cfg --misc tegra234-mb1-bct-misc-p3701-0000_cpp.dtb --device tegra234-mb1-bct-device-p3701-0000_cpp.dtb
[   0.1991 ] MB1-BCT version: 0.9
[   0.2011 ] Parsing config file :tegra234-mb1-bct-device-p3701-0000_cpp.dtb 
[   0.2013 ] Added Platform Config 9 data with size :- 80
[   0.2013 ] 
[   0.2014 ] Updating mb1-bct with firmware information
[   0.2027 ] tegrabct_v2 --chip 0x23 0 --mb1bct mb1_cold_boot_bct_MB1.bct --updatefwinfo readinfo_t234_min_prod.xml.bin
[   0.2056 ] tegrahost_v2 --chip 0x23 0 --align mb1_cold_boot_bct_MB1_aligned.bct
[   0.2082 ] tegrahost_v2 --chip 0x23 0 --magicid MBCT --appendsigheader mb1_cold_boot_bct_MB1_aligned.bct zerosbk
[   0.2092 ] adding BCH for mb1_cold_boot_bct_MB1_aligned.bct
[   0.2118 ] tegrasign_v3.py --key None --list mb1_cold_boot_bct_MB1_aligned_sigheader.bct_list.xml --pubkeyhash pub_key.key --sha sha512
[   0.2120 ] Assuming zero filled SBK key
[   0.2140 ] Warning: pub_key.key is not found
[   0.2136 ] tegrahost_v2 --chip 0x23 0 --updatesigheader mb1_cold_boot_bct_MB1_aligned_sigheader.bct.encrypt mb1_cold_boot_bct_MB1_aligned_sigheader.bct.hash zerosbk
[   0.2150 ] Generating recovery mb1-bct
[   0.2158 ] tegrabct_v2 --chip 0x23 0 --mb1bct mb1_bct.cfg --misc tegra234-mb1-bct-misc-p3701-0000_cpp.dtb --device tegra234-mb1-bct-device-p3701-0000_cpp.dtb
[   0.2165 ] MB1-BCT version: 0.9
[   0.2176 ] Parsing config file :tegra234-mb1-bct-device-p3701-0000_cpp.dtb 
[   0.2178 ] Added Platform Config 9 data with size :- 80
[   0.2178 ] 
[   0.2178 ] Updating mb1-bct with firmware information
[   0.2186 ] tegrabct_v2 --chip 0x23 0 --mb1bct mb1_bct_MB1.bct --recov --updatefwinfo readinfo_t234_min_prod.xml.bin
[   0.2207 ] tegrahost_v2 --chip 0x23 0 --align mb1_bct_MB1_aligned.bct
[   0.2220 ] tegrahost_v2 --chip 0x23 0 --magicid MBCT --appendsigheader mb1_bct_MB1_aligned.bct zerosbk
[   0.2226 ] adding BCH for mb1_bct_MB1_aligned.bct
[   0.2248 ] tegrasign_v3.py --key None --list mb1_bct_MB1_aligned_sigheader.bct_list.xml --pubkeyhash pub_key.key --sha sha512
[   0.2250 ] Assuming zero filled SBK key
[   0.2271 ] Warning: pub_key.key is not found
[   0.2264 ] tegrahost_v2 --chip 0x23 0 --updatesigheader mb1_bct_MB1_aligned_sigheader.bct.encrypt mb1_bct_MB1_aligned_sigheader.bct.hash zerosbk
[   0.2277 ] Error: Skip generating mem_bct because sdram_config is not defined
[   0.2277 ] Error: Skip generating mem_bct because sdram_config is not defined
[   0.2277 ] Copying signatures
[   0.2287 ] tegrahost_v2 --chip 0x23 0 --partitionlayout readinfo_t234_min_prod.xml.bin --updatesig images_list_signed.xml
[   0.2343 ] mb1_t234_prod_aligned_sigheader.bin.encrypt filename is from images_list
[   0.2344 ] psc_bl1_t234_prod_aligned_sigheader.bin.encrypt filename is from images_list
[   0.2344 ] Boot Rom communication
[   0.2352 ] tegrarcm_v2 --new_session --chip 0x23 0 --uid --download bct_br br_bct_BR.bct --download mb1 mb1_t234_prod_aligned_sigheader.bin.encrypt --download psc_bl1 psc_bl1_t234_prod_aligned_sigheader.bin.encrypt --download bct_mb1 mb1_bct_MB1_sigheader.bct.encrypt
[   0.2358 ] BR_CID: 0x80012344705E35E03C00000014008240
[   0.2368 ] Sending bct_br
[   0.2550 ] ERROR: might be timeout in USB write.
=====
2

I connected UART and it seems not able to boot up. Would this be a hardware issue?

Jetson UEFI firmware (version 2.1-32413640 built on 2023-01-24T23:12:27+00:00)
ESC   to enter Setup.
F11   to enter Boot Manager Menu.
Enter to continue boot.
**  WARNING: Test Key is used.  **
....
    L4TLauncher: Attempting GRUB Boot
L4TLauncher: Attempting Direct Boot
EFI stub: Booting Linux Kernel...
EFI stub: Using DTB from configuration table
EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path
EFI stub: Exiting boot services and installing virtual address map...
3

I found a work around. Had to flash to an external nvme drive, and with that I’m able to boot up.

5

Hi Colin,

Are you using standalone Ubuntu 20.04 as your host PC?

Do you have NVMe drive on your devkit?
How about removing NVMe drive and flashing into internal eMMC?

6

Do you have NVMe drive on your devkit?
Yes, the NVMe drive is attached with the devkit though the 2280 slot under the box.

Are you using standalone Ubuntu 20.04 as your host PC?
Before adding NVMe drive to the devkit, I tried both physical Ubuntu 20.04 PC and Ubuntu 20.04 virtual machine (a VM from VMware workstation on a Windows 11 PC), and both failed to flash internal eMMC. Then I added NVMe drive, used the Ubuntu 20.04 VM to flash the NVMe, and the devkit was able to boot.

How about removing NVMe drive and flashing into internal eMMC?
I’ll do that and let you know the results.

7

Please just use standalone Ubuntu 20.04 PC for SDK manager to get stable connection with the board while flashing.

8

I removed NVMe drive and used standalone Ubuntu 20.04 PC but flash still failed.

9

This issue looks like some other users got on Orin.
Do you have Ubuntu 18.04 host PC to flash the board with SDK manager?

10

Yes I can find a Ubuntu 18.04 to give it a try.

But, I want to emphasize that the flash issue may have something to do with the Jetson Linux upgrade. Everything was running fine with 35.1 and then the box was unable to boot up after upgrade to 35.2. I didn’t worry about the data so I just went on to re-flash the eMMC, and that was when the flash issue happened. The root cause might be related to the upgrade, which seems to cause the eMMC unrecognizable.

11

Please let us know the result, thank you.

How did you upgrade from 35.1 to 35.2?

eMMC should be recognized because you could boot up with NVMe drive, there should be some boot contents in eMMC for boot up. NVMe is only used for rootfs.

12

Please let us know the result, thank you.

Amazingly SDK Manager in Ubuntu 18.04 is able to flash the internal eMMC. Now the box is able to boot up with root pointing to /dev/mmcblk0p1.

How did you upgrade from 35.1 to 35.2?

I did update and dist-upgrade, but the box still booted up to 35.1, so I changed file /etc/apt/sources.list.d/nvidia-l4t-apt-source.list to point the repository to 35.2 and this time upgrade was able to install 35.2 packages, but it failed to boot up.

13

Could you provide the detail (brand, model) of your host PC?
There are several users got the similar results as you, we want collect more information.

Could you provide the full serial console log at this moment for further check?
There’s just UEFI part above.

14

Could you provide the detail (brand, model) of your host PC?

I don’t have a physical Ubuntu 18.04 machine so I used a virtual machine to flash. The VM is in a DELL XPS 8940 desktop box. The cable for flash is a USB-A to USB-C, with A side connected to host and C side to Orin. The SDK Manager is the latest 1.9.1.

Could you provide the full serial console log at this moment for further check?

Boot log through serial console is attached.

orin-boot-log.txt (58.2 KB)

15

Thanks for your information.

17

Are you sure what the problem is? I used 18.0.4, but it also failed

18

please run this on your host PC. Try to reproduce the error again and share us the dmesg on your host.

echo 8 > /proc/sys/kernel/printk
echo ‘module usbcore +p’ > /sys/kernel/debug/dynamic_debug/control
echo ‘module xhci_hcd +p’ > /sys/kernel/debug/dynamic_debug/control

19

The system that I burned AGX-ORIN-NX-32GB can start, but it will report an error

023/4/3 9:59:17.509] I> FSITHERM max temp = 33
[2023/4/3 9:59:17.515] I> FSITHERM min temp = 32
[2023/4/3 9:59:17.515] I> Task: Restore XUSB sec (0x50001d44)
[2023/4/3 9:59:17.524] I> Task: Enable FSI SE clock (0x5001606c)
[2023/4/3 9:59:17.524] I> Enable FSI-SE clock and 10ms delay…
[2023/4/3 9:59:17.536] I> Task: Initialize SBSA UART CAR (0x50001f48)
[2023/4/3 9:59:17.541] I> Task: Initialize CPUBL Params (0x50016f88)
[2023/4/3 9:59:17.541] I> CPUBL-params @ 0x1036000000
[2023/4/3 9:59:17.549] I> Task: Prepare eeprom data (0x50016d54)
[2023/4/3 9:59:17.575] E> I2C: slave not found in slaves.
[2023/4/3 9:59:17.582] E> I2C: Could not write 0 bytes to slave: 0x00ac with repeat start true.
[2023/4/3 9:59:17.590] E> I2C_DEV: Failed to send register address 0x00000000.
[2023/4/3 9:59:17.597] E> I2C_DEV: Could not read 256 registers of size 1 from slave 0xac at 0x00000000 via instance 0.
[2023/4/3 9:59:17.605] E> eeprom: Failed to read I2C slave device
[2023/4/3 9:59:17.605] C> Task 0x0 failed (err: 0x1f1e050d)
[2023/4/3 9:59:17.611] E> Top caller module: I2C_DEV, error module: I2C, reason: 0x0d, aux_info: 0x05
[2023/4/3 9:59:17.611] I> Busy Spin

20

So which system should I use??? For AGX-ORIN-64GB module

21

Hi @1874077537

Your issue is different from this post. Please file your new topic.

22

It’s like this. For the AGX-ORIN-64GB module, I have burned two systems in total. One is the AGX-ORIN development kit (AGX-ORIN-32GB module) system, and the other is the system that selects the AGX-ORIN-64GB module under the SDK, which will report errors。-_-