Failed to boot : CPU Bootloader is not running on device

My Jetson Xavier NX is not booting anymore. I wrote a new image (Jetson Xavier NX Developer Kit SD Card Image) on a brand new micro SD card , and I am still stuck on the same screen when I boot the jetson Xavier NX
“bash: cannot set terminal process group (-1): Inappropriate ioctl for device” , so I believe my only solution is to re-flash with jetpack. Yes I am booting it on recovery mode.

Flashing the device is failing with this :

aya@ubuntu:/media/aya/new100gb/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra$ sudo ./flash.sh jetson-xavier-nx-devkit mmcblk0p1
[sudo] password for aya:
###############################################################################

L4T BSP Information:

R32 , REVISION: 4.2

###############################################################################

Target Board Information:

Name: jetson-xavier-nx-devkit, Board Family: t186ref, SoC: Tegra 194,

OpMode: production, Boot Authentication: NS,

###############################################################################
copying soft_fuses(/media/aya/new100gb/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra/bootloader/t186ref/BCT/tegra194-mb1-soft-fuses-l4t.cfg)… done.
./tegraflash.py --chip 0x19 --applet “/media/aya/new100gb/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra/bootloader/mb1_t194_prod.bin” --skipuid --soft_fuses tegra194-mb1-soft-fuses-l4t.cfg --bins “mb2_applet nvtboot_applet_t194.bin” --cmd “dump eeprom boardinfo cvm.bin;reboot recovery”
Welcome to Tegra Flash
version 1.0.0
Type ? or help for help and q or quit to exit
Use ! to execute system commands

[ 0.0047 ] Generating RCM messages
[ 0.0117 ] tegrahost_v2 --chip 0x19 0 --magicid MB1B --appendsigheader /media/aya/new100gb/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra/bootloader/mb1_t194_prod.bin zerosbk
[ 0.0126 ] Header already present for /media/aya/new100gb/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra/bootloader/mb1_t194_prod.bin
[ 0.0196 ]
[ 0.0240 ] tegrasign_v2 --key None --getmode mode.txt
[ 0.0252 ] Assuming zero filled SBK key
[ 0.0292 ]
[ 0.0311 ] tegrasign_v2 --key None --file /media/aya/new100gb/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra/bootloader/mb1_t194_prod_sigheader.bin --offset 2960 --length 1136 --pubkeyhash pub_key.key
[ 0.0320 ] Assuming zero filled SBK key
[ 0.0329 ]
[ 0.0345 ] tegrahost_v2 --chip 0x19 0 --updatesigheader /media/aya/new100gb/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra/bootloader/mb1_t194_prod_sigheader.bin /media/aya/new100gb/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra/bootloader/mb1_t194_prod_sigheader.hash zerosbk
[ 0.0391 ]
[ 0.0418 ] tegrabct_v2 --chip 0x19 0 --sfuse tegra194-mb1-soft-fuses-l4t.cfg.pdf sfuse.bin
[ 0.0463 ]
[ 0.0473 ] tegrarcm_v2 --listrcm rcm_list.xml --chip 0x19 0 --sfuses sfuse.bin --download rcm /media/aya/new100gb/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra/bootloader/mb1_t194_prod_sigheader.bin 0 0
[ 0.0481 ] RCM 0 is saved as rcm_0.rcm
[ 0.0504 ] RCM 1 is saved as rcm_1.rcm
[ 0.0504 ] RCM 2 is saved as rcm_2.rcm
[ 0.0504 ] List of rcm files are saved in rcm_list.xml
[ 0.0504 ]
[ 0.0504 ] Signing RCM messages
[ 0.0511 ] tegrasign_v2 --key None --list rcm_list.xml --pubkeyhash pub_key.key --getmontgomeryvalues montgomery.bin
[ 0.0517 ] Assuming zero filled SBK key
[ 0.0523 ]
[ 0.0523 ] Copying signature to RCM mesages
[ 0.0530 ] tegrarcm_v2 --chip 0x19 0 --updatesig rcm_list_signed.xml
[ 0.0546 ]
[ 0.0547 ] Boot Rom communication
[ 0.0555 ] tegrarcm_v2 --chip 0x19 0 --rcm rcm_list_signed.xml --skipuid
[ 0.0564 ] RCM version 0X190001
[ 0.3792 ] Boot Rom communication completed
[ 1.3984 ]
[ 2.4026 ] tegrarcm_v2 --isapplet
[ 2.4041 ] Applet version 01.00.0000
[ 3.0202 ]
[ 3.0213 ] tegrarcm_v2 --ismb2
[ 3.6744 ]
[ 3.6756 ] tegrahost_v2 --chip 0x19 --align nvtboot_applet_t194.bin
[ 3.6777 ]
[ 3.6787 ] tegrahost_v2 --chip 0x19 0 --magicid PLDT --appendsigheader nvtboot_applet_t194.bin zerosbk
[ 3.6797 ] adding BCH for nvtboot_applet_t194.bin
[ 3.6829 ]
[ 3.6841 ] tegrasign_v2 --key None --list nvtboot_applet_t194_sigheader.bin_list.xml --pubkeyhash pub_key.key
[ 3.6850 ] Assuming zero filled SBK key
[ 3.6853 ]
[ 3.6902 ] tegrahost_v2 --chip 0x19 0 --updatesigheader nvtboot_applet_t194_sigheader.bin.encrypt nvtboot_applet_t194_sigheader.bin.hash zerosbk
[ 3.6931 ]
[ 3.6941 ] tegrarcm_v2 --download mb2 nvtboot_applet_t194_sigheader.bin.encrypt
[ 3.6950 ] Applet version 01.00.0000
[ 4.3491 ] Sending mb2
[ 4.3492 ] […] 100%
[ 4.3688 ]
[ 4.3699 ] tegrarcm_v2 --boot recovery
[ 4.3706 ] Applet version 01.00.0000
[ 5.0402 ]
[ 6.0436 ] tegrarcm_v2 --isapplet
[ 6.6987 ]
[ 6.6997 ] tegrarcm_v2 --ismb2
[ 6.7007 ] MB2 Applet version 01.00.0000
[ 7.3563 ]
[ 7.3574 ] tegrarcm_v2 --ismb2
[ 7.3583 ] MB2 Applet version 01.00.0000
[ 8.0253 ]
[ 8.0269 ] Retrieving EEPROM data
[ 8.0271 ] tegrarcm_v2 --oem platformdetails eeprom cvm /media/aya/new100gb/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra/bootloader/cvm.bin
[ 8.0282 ] MB2 Applet version 01.00.0000
[ 8.6844 ] 0000000036360018:
[ 8.7462 ]
[ 8.7491 ] tegradevflash_v2 --oem platformdetails eeprom cvm /media/aya/new100gb/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra/bootloader/cvm.bin
[ 8.7500 ] CPU Bootloader is not running on device.
[ 9.0756 ]
Error: Return value 4
Command tegradevflash_v2 --oem platformdetails eeprom cvm /media/aya/new100gb/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra/bootloader/cvm.bin
Reading board information failed.

If I retry the error message is shorter :

aya@ubuntu:/media/aya/new100gb/JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT/Linux_for_Tegra$ sudo ./flash.sh jetson-xavier-nx-devkit mmcblk0p1
###############################################################################

L4T BSP Information:

R32 , REVISION: 4.2

###############################################################################
Error: probing the target board failed.
Make sure the target board is connected through
USB port and is in recovery mode.

I am using a data USB cable , and as I only have a MacBook pro , I tried with Virtualbox with ubuntu 16 and ubuntu 18 , and with vmware fusion with ubuntu 16 and 18 , and with USB 2 and USB 3 , bridged or wifi network, nothing worked.

Any idea ?

thanks.

1 Like

Do you have VirtualBox set up to automatically connect the NX in recovery mode to the VM?

Oh, you may want to try running Docker with a Ubuntu container since it has fewer layers to get from the image to the USB device.

that’s what I tried last

It’s not working either with usb2 or without filter or by connecting it after VM boot …

You mean a docker image running on macOS ? How would I install the nvidia sdkmanager without gui ?

Don’t. Download the L4T Driver package from https://developer.nvidia.com/embedded/L4T/r32_Release_v4.2/t186ref_release_aarch64/Tegra186_Linux_R32.4.2_aarch64.tbz2

No GUI. You just run the same ./flash.sh command. In fact, you may want to download it in the VM and try it in case there’s something goofy with the SDK manager.

1 Like

do you have an idea how to share the usb port with docker ?
here is what I have when I lsusb on the mac :
Bus 020 Device 044: ID 0955:7e19 NVIDIA APX Serial: 00000

but when I ls the /dev/tty* I have this

so I don’t know how to change this command

docker run -t -i --device=/dev/ttyUSB0 ubuntu bash

with my USB port containing the Nvidia …

In recovery mode, the NX doesn’t show up as any normal device so…
When the NX registers on the Mac, in lsusb you should see something like

Bus 005 Device 123: ID 0955:7f21 NVIDIA Corp. 4-Port USB 2.0 Hub

When you start the docker container, use the Bus and Device to construct the device path…

docker run -it --device bus/usb/005/123 ubuntu bash

When you then run flash.sh…

./flash.sh --usb-instance bus/usb/005/123 etc,

Oh, if you go back to your VM and do an lsusb get the bus and device and see if specifying it on the flash command line helps.

So on the VM , I have
Bus 003 Device 005: ID 0955:7e19 NVidia Corp.

running :
sudo ./flash.sh --usb-instance bus/usb/003/005 jetson-xavier-nx-devkit mmcblk0p1

fail with
[ 3.3171 ] CPU Bootloader is not running on device.
[ 3.3268 ]
Error: Return value 4

On the mac, with docker , when I try
docker run -it --device bus/usb/020/026 ubuntu bash

it fail with
docker: bus/usb/020/026 is not an absolute path.

I even tried to share the usb port with NOMACHINE with my lab computer 8000km from here which is running native ubuntu 18 , but it also failed with the same error message (even though percentage showed 50% , don’t know why maybe it was just slower…)

Ah, I mistyped the docker command. The device should be /dev/bus/usb/…

thx for trying, didn’t work. I tried it on virtualbox and on vmware and nomachine, same error each time : CPU Bootloader is not running on device
I also tried with a third USB cable, same.

on mac there is no /dev/usb , it says
docker: Error response from daemon: error gathering device information while adding custom device “/dev/bus/000/030”: no such file or directory.

what is strange is that the first time after reboot on recovering mode the jetson and trying the flash.sh , the sequence is significatively longer before it fail, but after that , each try with flash.sh is only 3 line (see my first post), so I guess it is communicating somehow with the board …

EDIT : I also tried Parallels Desktop with Ubuntu 18 , with USB2 and USB3 , no luck, exactly the same…

Frustrating I’ll bet. One thing I noticed in my testing yesterday is that any attempt that is even partially successful leaves the NX in a state where you can’t continue without powering off then powering on again in recovery mode.

I did find this blog post…

I didn’t realize that on Mac Docker runs in a VM. Maybe that blog post will help although if vbox didn’t work on it’s own maybe it won’t work here either.

So I finally got my hand on a PC laptop, I booted it under Ubuntu 18 and tried flash.sh , same result and same error message. Guess I don’t have other option that to return it to nvidia and buy another one …
If you wander how I got there , it was an apt-get update and upgrade , I was SSHing it so I got the trace of the suspicious line :

  1. Updating certificates in /etc/ssl/certs…
  2. 2 added, 8 removed; done.
  3. Setting up libexif12:arm64 (0.6.21-4ubuntu0.5) …
  4. Setting up linux-firmware (1.173.18) …
  5. update-initramfs: Generating /boot/initrd.img-4.9.140-tegra
  6. Warning: couldn’t identify filesystem type for fsck hook, ignoring.
  7. I: The initramfs will attempt to resume from /dev/zram1

and this :

  1. Setting up gir1.2-nm-1.0:arm64 (1.10.6-2ubuntu1.4) …
  2. Setting up nvidia-l4t-bootloader (32.4.2-20200429224812) …
  3. 3668----1–jetson-xavier-nx-devkit-mmcblk0p1
  4. Starting bootloader post-install procedure.
  5. Update bootloader completed.
  6. Reboot the target system for changes to take effect.

I don’t know why the hell the dist-upgrade was updating my firmware and my bootloader.
I was trying to update OPENCV to get the camera work with python3 …

dist-upgrade is a confusing alias for full-upgrade. (Confusing because it sounds like it might upgrade your distro to the next version, but it never does this).

The OTA updates can update the bootloader, but that shouldn’t break your system. If I were you I would wait for a response from Nvidia before returning the device, since they may be able to suggest something.

In any case, If it is a issue that can’t be fixed, you can RMA it it directly to Nvidia and they will send you a new one. If somehow the bootloader update from Nvidia did to break the system they may want to do a post mortem anyway. That really shouldn’t happen.

1 Like

So, if you hook up a USB serial adapter to the UART TXD and UART RXD pins on the NX’s button header and reboot as normal, what do you get? Depending on the error and where it happens, there may be an alternative. The output will be a little different than your original post with the display connected.

1 Like

I got the same issue today and can’t even trust that @iliesaya reach the same issue as mine.

We use a custom docker based image to flash our Xavier NX over custom baseboard. Our rootfs was customized for a previous JP 35.4.5 and all nvidia .deb was for this version.

Our Dockerfile just take the latest JP 35.5.2 and rootfs mentioned above and proceed to build our kernel, dtb, build custom drivers, update pinmux, customize l4t_initrd and then expose flash.sh. - pls note without applying binaries as the rootfs was already prepared with .deb version from 32.4.5.

The rootfs get the /etc/apt/source… configured with t194 r32.5.

Flash was working everytime during days w/o issues using this method and finally we decided to update the latest .deb as @iliesaya did through apt update/upgrade.

While doing it I noticed clearly at this time the nvidia-l4t-bootloader update… After rebooting, the Xavier NX was not starting, I tried many times to switch to recovery mode and re-flash and got the same error mentioned by @iliesaya

Nvidia ? Pls advice on this and confirm we need to RMA ?

1 Like

File a new topic please. This thread has been closed for one year.