Unable to flash Jetson Orin Nano

I am running Ubuntu 22.04.3 LTS (GNU/Linux 5.15.133.1-microsoft-standard-WSL2 x86_64) trying to flash an 8GB Jetson Orin Nano device with the latest SDK using sdkmanager on Windows 11 WSL. I have the device set in recovery mode via jumpers, it is attached using WSL commands, and I can see the device inside the WSL instance. I get about 25% into flashing and it errors out. I also notice the usb ejection noise coming from the machine 7 or 8 times during the flashing process and a “WSL usbip: error: Attach Request for 1-1 failed - Device not found” message in the usbipd attach PS window.

Tried attaching with usb vid/pid instead of bus id, but the problem persists.

[ 0.0863 ] BR_CID: 0x80012344705DE5C32000000018FE0200
[ 0.0887 ] Sending bct_br
[ 0.0887 ] ERROR: might be timeout in USB write.
Error: Return value 3
Command tegrarcm_v2 --new_session --chip 0x23 0 --uid --download bct_br br_bct_BR.bct --download mb1 mb1_t234_prod_aligned_sigheader.bin.encrypt --download psc_bl1 psc_bl1_t234_prod_aligned_sigheader.bin.encrypt --download bct_mb1 mb1_bct_MB1_sigheader.bct.encrypt
20:31:18.421 - Error: [exec_command]: /bin/bash -c /tmp/tmp_NV_L4T_FLASH_JETSON_LINUX_COMP.lilhoser.sh; [error]: *** ERROR: Parsing boardid failed
SDKM_logs_JetPack_6.0_DP_Linux_for_Jetson_Orin_Nano_modules_2024-01-23_20-29-50.zip (109.8 KB)

You need this:

, and this:

You won’t be able to flash the device with the default WSL2 setting even after the timeout issue is solved, so please consider finding a real Ubuntu PC instead unless you really cannot.

I appreciate the links.

The first FAQ you linked did not work:

(base) root@PAXAN:/# echo -1 > /sys/module/usbcore/parameters/autosuspend
bash: /sys/module/usbcore/parameters/autosuspend: Permission denied

And the second FAQ you linked states “For flashing eMMC/SD card with flash.sh, the default setting of WSL2 will work fine.” I am trying to flash an SD card and it is definitely not working fine per the logs I posted.

I now have a $500 brick

I missed it.
WSL2 does not by default enable any dynamic kernel modules; instead, everything is built into the kernel image.
Therefore, you have to specify this parameter for disabling USB auto suspend through kernel command lines, which can be done with WSL2 config files.
Some links for your reference:

https://www.kernel.org/doc/html/v5.0/admin-guide/kernel-parameters.html

Maybe I should have made it more clear.
SDK Manager uses initrd flash to flash Orin Nano with SD cards, which is the official way with Orin NX/Nano, but you can also use flash.sh, which uses different mechanisms from initrd flash, and not does require a custom WSL2 kernel.

sudo ./flash.sh jetson-orin-nano-devkit internal

I’m not sure if WSL2 kernel supports the autosuspend parameter for usbcore. I tried setting this kernel command line parameter in the global .wslconfig, which it accepted, but the parameter does not exist in /sys/module/usbcore/parameters.

[wsl2]

kernelCommandLine= usbcore.autosuspend=-1

Using flash.sh does not work:

lilhoser@PAXAN:/mnt/c/Downloads/Jetson_Linux_R36.2.0_aarch64/Linux_for_Tegra$ sudo ./flash.sh jetson-orin-nano-devkit internal
[sudo] password for lilhoser:
Error: CHIPID of  is not supported.
1 Like

It’s not a loadable kernel modules so it’s expected it won’t appear there.

Don’t put the BSP in a Windows folder. The permission issue between Windows and Linux may cause a lot of unexpected errors.
Also make sure the device is detected with lsusb in the WSL2 environment.

The Jetson device does appear in the lsusb output.

I moved the BSP into /home/lilhoser and I get the same error as above.

Can you please delete the BSP folder, download it again manually, and setup according to the document?
https://docs.nvidia.com/jetson/archives/r36.2/DeveloperGuide/IN/QuickStart.html

Ok, I took your advice and went out and bought a new miniPC to install Linux on. Guess what, it still doesn’t work.

It downloaded all of the packages for installation, got 95% through flashing the Jetson device, then failed spectacularly with cryptic messages sprayed all over the log. Here are a few:

16:37:36.245 - error: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS: tar: Write checkpoint 10000
16:37:36.245 - info: Event: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS - error is: tar: Write checkpoint 10000
16:37:36.589 - error: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS: tar: Write checkpoint 20000
16:37:36.589 - info: Event: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS - error is: tar: Write checkpoint 20000
16:37:36.886 - error: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS: tar: Write checkpoint 30000
16:37:36.886 - info: Event: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS - error is: tar: Write checkpoint 30000
16:37:37.168 - error: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS: tar: Write checkpoint 40000
16:37:37.168 - info: Event: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS - error is: tar: Write checkpoint 40000
16:37:37.482 - info: NV_CUDA_HOST_COMP@host: Selecting previously unselected package cuda-nsight-systems-12-2.

And

16:37:58.145 - info: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS: [   0.0181 ] tegrarcm_v2 --chip 0x23 0 --ismb2applet
16:37:58.145 - info: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS: [   0.0184 ] File rcm_state open failed
16:37:58.150 - error: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS: [   0.0186 ] ERROR: failed to read rcm_state
16:37:58.150 - info: Event: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS - error is: [   0.0186 ] ERROR: failed to read rcm_state

and

16:37:58.291 - error: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS: [   0.1309 ] Error: Skip generating mem_bct because sdram_config is not defined
16:37:58.291 - info: Event: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS - error is: [   0.1309 ] Error: Skip generating mem_bct because sdram_config is not defined
16:37:58.291 - error: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS: [   0.1309 ] Error: Skip generating mem_bct because sdram_config is not defined
16:37:58.291 - info: Event: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS - error is: [   0.1309 ] Error: Skip generating mem_bct because sdram_config is not defined

and

16:38:07.884 - error: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS: 20+0 records in
16:38:07.884 - info: Event: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS - error is: 20+0 records in
16:38:07.884 - error: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS: 20+0 records out
16:38:07.884 - info: Event: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS - error is: 20+0 records out
16:38:07.884 - error: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS: 20 bytes copied, 0.00014307 s, 140 kB/s
16:38:07.884 - info: Event: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS - error is: 20 bytes copied, 0.00014307 s, 140 kB/s
16:38:07.888 - error: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS: /home/lilhoser/nvidia/nvidia_sdk/JetPack_6.0_DP_Linux_DP_JETSON_ORIN_NANO_TARGETS/Linux_for_Tegra/flash.sh: line 3145: python: command not found

Now all subsequent attempts to run sdkmanager throw this:

The screenshot showing sdkmanager not able to connect is because, for whatever reason, the jeston device booted up and was stuck at a kernel panic. I didn’t think the jetson device would boot when set into recovery mode via the jumpers? Anyway, I powered it off, removed and added the jumper again, and now sdkmanager can connect.

I began flashing it again, and about halfway through, I get the installation failed “cannot connect to target device” error and again the Jetson device seems to have booted itself up into a kernel panic.

While flashing this time and the last time, I get a bunch of prompts about “Nvidia (ADX) added” and “removed”. Kernel grub arguments are set to disable usb autosuspend (-1).

I find this quite odd:

16:37:36.245 - error: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS: tar: Write checkpoint 10000
16:37:36.245 - info: Event: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS - error is: tar: Write checkpoint 10000
16:37:36.589 - error: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS: tar: Write checkpoint 20000
16:37:36.589 - info: Event: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS - error is: tar: Write checkpoint 20000
16:37:36.886 - error: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS: tar: Write checkpoint 30000
16:37:36.886 - info: Event: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS - error is: tar: Write checkpoint 30000
16:37:37.168 - error: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS: tar: Write checkpoint 40000
16:37:37.168 - info: Event: NV_L4T_FLASH_JETSON_LINUX_COMP@JETSON_ORIN_NANO_TARGETS - error is: tar: Write checkpoint 40000

A tar error means the content being unpacked is corrupt. Typically I would think that is either a corrupt download, or else the filesystem itself which you are unpacking into has problems (maybe as simple as insufficient space). On any host PC where you’ve run this, what do you see from:
df -H -T

There is a possibility that a tar file within the download was corrupt before the download, but it also looks like your host PC is missing python. If you try to add these, do either of them install? And if they install, does the tar error go away?

sudo apt-get install python2-minimal
sudo apt-get install python3-minimal
Filesystem     Type      Size  Used Avail Use% Mounted on
tmpfs          tmpfs     6.7G  2.5M  6.7G   1% /run
/dev/nvme0n1p5 ext4      809G   73G  696G  10% /
tmpfs          tmpfs      34G     0   34G   0% /dev/shm
tmpfs          tmpfs     5.3M  4.1k  5.3M   1% /run/lock
efivarfs       efivarfs  132k   41k   86k  32% /sys/firmware/efi/efivars
/dev/nvme0n1p1 vfat      101M   62M   39M  62% /boot/efi
tmpfs          tmpfs     6.7G  119k  6.7G   1% /run/user/1000

I noticed the python error and python 3 is already installed, which on ubuntu requires using python3 and not python (seems like this is a bug in sdkmanager or whatever script it’s calling). To prevent this again, I installed the python3-is-python lib.

I deleted everything sdkmanager downloaded and tried to follow these instructions manually: Quick Start — NVIDIA Jetson Linux Developer Guide 1 documentation

It seemed to flash the jetson, but then it died with “waiting for target to boot up”. Logs:

output.txt (259.9 KB)

Switching to the Jetson, it booted up into the same kernel panic I mentioned in my prior post:

Can you get a serial console log of the entire boot and attach that to the forum? The attached image is useful, but an actual log would be far more useful. More on that below.

I see the host PC filesystem is ok and not an issue. Having ext4 filesystem type and sufficient space is what I was looking for. Also, you are getting tar errors. I don’t know what WSL2 is doing to trigger this. It seems tar checkpoints are a customization, and there isn’t much of a way to tell what the checkpoint test was. It might be as simple as just an echo of the checkpoint being reached. This is rather rare to see, and it was possible the host PC filesystem was at fault, but most likely it is the tar file itself. There is also a chance that system RAM is at fault, but I’d consider that a tiny chance unless WSL2 is limiting something.

This might not matter, but there was an earlier error of not finding Python. Just for reference, which release is shown from each of these?

  • python --version
  • python2 --version
  • python3 --version

(Someone from NVIDIA might know if using just the “python” command is required to be version 2.x or 3.x python; if not, then it might still be useful to know which version is present with each command)

When you say “latest” version of SDK Manager, do you mean the version tied to L4T R36.x developer preview? If you go to your “Linux_for_Tegra/rootfs/etc/” you can run command “head -n 1 nv_tegra_release” to see the L4T release being flashed. The reason I ask is that you are using Ubuntu 22.04, and L4T 6.x is possible with this, but L4T R5.x is not. L4T R6.x is considered a developer preview, whereas 5.x is considered the stable release.

Incidentally, it looks like preparation for flash was successful, but there was no recognition of the Jetson being present in recovery mode. If you ignore the tar checkpoint errors, you still get to the point where everything is set for flash, and then the start of flash does not occur because a recovery mode Jetson is not seen. Almost all cases of this when using a VM is the VM setup being incorrect and not passing the recovery mode Jetson through to the Ubuntu o/s. During a flash the Jetson has a disconnect and reconnect, and sometimes (A) the first connect does not occur because of the VM setup, or sometimes (B) the initial connect works, but the reconnect fails. It looks like the former case in your log, that the Jetson was never visible to the VM.

Just for the record, if the recovery pin is held low (shorted to ground) while either of these occur, and then the pin is unshorted (there is no delay required), the Jetson should be in recovery mode:

  • Power on.
  • Power reset.

Once the recovery mode Jetson is present the Linux host will see Jetson devices. The command to see any Jetson would be “lsusb -d '0955:'”. That lists all NVIDIA USB devices. The Orin should be the product ID 7023, so if you wanted to limit this to Orin you would use “lsusb -d 0955:7023”. Before flashing, if your Jetson is in recovery mode, does the WSL2 see this device via “lsusb -d '0955:'”?

Incidentally, although I suspect something in WSL2, your screenshot shows that the Jetson booted. Possibly the QSPI memory failed to flash something which supports the release on any SD card (or other media). I saw a note of initrd flash, but I’ve not seen a mention of whether you are using external media, e.g., an SSD or NVMe. Is this purely an SD card flash, or is there other media involved?

In the case of the image the error I see is in exception level 0 (that’s the el0_). This is user space, and so it isn’t a driver causing that, it is something in user land. Previous to this there is a note about missing init. This is absolutely not something that can succeed, it implies that the kernel loaded and began its first task (init is in fact process ID 1, whereas the kernel is PID 0; init is a symbolic link in Ubuntu to systemd…this is the source of all processes, whereas the kernel services the system calls to the processes). In other words, whatever your root filesystem was specified as, the kernel loaded as it should, but the filesystem was missing the most important program there is (the first user space process).

If you performed some ordinary flash on an SD card model of Orin dev kit, then it would be flashing QSPI and be set up to find that filesystem (and init) on the SD card. The QSPI content would have to be the same major release as that of the SD card. A JetPack 5/L4T R35.x QSPI flash can’t be used with a JetPack 6/L4T R36.x L4T SD card; the reverse is also true, you can’t flash JP5/L4T R36.x to QSPI and use it with a JP 5/L4T R35.x SD card. Being that the host PC is Ubuntu 22.04 (if WSL2 is working), then I would expect all content must be JP 6, including both the QSPI flash and the SD card.

If you are using external media and not purely SD card, then you would have to flash using different methods. This is why an initrd flash is used. That initrd flash acts as an adapter of sorts between a normal QSPI+SD flash to allow the rootfs to be on other media. If something were mixed in there which was not consistent on whether this was set up for external media, then I could see minimal content existing which would find and load the kernel, but then try to pivot to the external media filesystem and fail to find init. That’s just guessing, but it seems likely that the issues revolve around flash procedure for external media (if you are using external media) and/or something WSL2 is failing.

Can you provide a detailed description of your external media, if it exists? Are you certain your Jetson is in recovery mode as evidenced by “lsusb -d '0955:'” prior to flash?

I am still reading through your post, but I wanted to clarify: I’m not using WSL2. In the original post here, I was using WSL2 and after banging my head on the wall for a week or so, I gave up and followed @DaveYYY 's advice in this post:

You won’t be able to flash the device with the default WSL2 setting even after the timeout issue is solved, so please consider finding a real Ubuntu PC instead unless you really cannot.

I am running sdkmanager from a physical Linux PC.

Then you won’t need to worry about whether USB drops out and fails to come back when there is a disconnect/reconnect. Everything else still applies, e.g., you need to see the recovery mode Jetson with “lsusb -d '0955:'” or “lsusb -d 0955:7023” (the former is any Jetson model, the latter is specific to Orin).

I’m not sure how to get the serial console log for the Jetson. If you can tell me how to do it, I’ll try.

lilhoser@omicrontheta:~$ python --version
Python 3.10.12
lilhoser@omicrontheta:~$ python2 --version
Python 2.7.18
lilhoser@omicrontheta:~$ python3 --version
Python 3.10.12

Here are the files I used:
Tegra_Linux_Sample-Root-Filesystem_R36.2.0_aarch64.tbz2
Jetson_Linux_R36.2.0_aarch64.tbz2

lilhoser@omicrontheta:~/Downloads/Linux_for_Tegra/rootfs/etc$ head -n 1 nv_tegra_release
# R36 (release), REVISION: 2.0, GCID: 34956989, BOARD: generic, EABI: aarch64, DATE: Thu Nov 30 19:03:58 UTC 2023

To answer your question, yes, I followed the instructions to verify my device was in recovery mode. But note, this is not accomplished with a push button, but rather a jumper setting beneath the GPU. output of ‘lsusb’ showed the Jetson device. After the unsuccessful flash, it does NOT appear in ‘lsusb’ output. If I hard reset it, it shows again:

Bus 007 Device 020: ID 0955:7523 NVIDIA Corp. APX

I am using the Nvidia Jetson Orin Nano developer board with an SD card. There is no other media involved. I am trying to flash Jetpack 6 onto this Jetson device, which is currently not bootable.

I am surprised to see “APX” in the lsusb output if not using a VM.

I only have the AGX Orin dev kit, which is ID 0955:7023. I’m guessing that :7523 product ID is for the Nano version, but if for some reason that ID is not what the flash software expects, then the device would not be found.

Can someone from NVIDIA verify if the Orin Nano lsusb should be 0955:7023 or if perhaps the 0955:7523 is causing SDK Manager to not see the recovery mode device? I don’t have the Orin Nano so I can’t verify what lsusb should show.

According to this article, 0955:7523 “APX” is correct:

Jetson Orin Nano - Flashing QSPI Firmware for More Memory - JetsonHacks

Can a mod please change the post title - this is not about WSL2. This is about the inability to flash a Jetson Orin Nano developer board as described in the last set of posts from February.

1 Like

Is this still an issue to support? Any result can be shared?