Stuck at kernel after apt update && apt upgrade

Hi there,

We are using 2 NVIDIA Jetson AGX Xavier at work.

The first one was flashed during april and the second one at begining of June.
Yesterday (2020/07/09) I did apt update && apt upgrade on the first one. But after reboot, I can only see the NVIDIA logo, then blinking cursor at the top left of the screen (kernel is loaded and then, does nothing)

Did the same on the second one, just to be sure lol.
Stuck at the same problem…

Having completely reflashed the second one, and cloned the resulting eMMC content to an USB Disk, I successfully booted the 1st one using eSATA/USB port.

The problem was in the extlinux.conf file

The one which doesn’t boot :

TIMEOUT 30
DEFAULT primary

MENU TITLE L4T boot options

LABEL primary
      MENU LABEL primary kernel
      LINUX /boot/Image
      INITRD /boot/initrd
      APPEND ${cbootargs} quiet

# When testing a custom kernel, it is recommended that you create a backup of
# the original kernel and add a new entry to this file so that the device can
# fallback to the original kernel. To do this:
#
# 1, Make a backup of the original kernel
#      sudo cp /boot/Image /boot/Image.backup
#
# 2, Copy your custom kernel into /boot/Image
#
# 3, Uncomment below menu setting lines for the original kernel
#
# 4, Reboot

# LABEL backup
#    MENU LABEL backup kernel
#    LINUX /boot/Image.backup
#    INITRD /boot/initrd
#    APPEND ${cbootargs}

The one that boots :

TIMEOUT 30
DEFAULT primary

MENU TITLE L4T boot options

LABEL primary
      MENU LABEL primary kernel
      LINUX /boot/Image
      INITRD /boot/initrd
      APPEND ${cbootargs} quiet root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyTCU0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 rootfstype=ext4 

# When testing a custom kernel, it is recommended that you create a backup of
# the original kernel and add a new entry to this file so that the device can
# fallback to the original kernel. To do this:
#
# 1, Make a backup of the original kernel
#      sudo cp /boot/Image /boot/Image.backup
#
# 2, Copy your custom kernel into /boot/Image
#
# 3, Uncomment below menu setting lines for the original kernel
#
# 4, Reboot

# LABEL backup
#    MENU LABEL backup kernel
#    LINUX /boot/Image.backup
#    INITRD /boot/initrd
#    APPEND ${cbootargs}

Not sure if :

  • the extlinux.conf file was actually right before the update (missing arguments were supposed to be somewhere else, before the update)
    or
  • the extlinux.conf file was messed up during the update (update deleted those arguments)

But what I’m sure about, is that in both case, it’s a problem ;)

Hope it will help
PS : not sure if it’s related, but on both of those Jetson AGX, an NVME SSD is installed (but not formatted and not used for the moment). I guess it’s not related but just in case I’m wrong, it’s better not to hide this detail.

Hi,
I am having the same problem. The black screen shows up and a blinking cursor at the top left of the screen.

Mine also has SSD.

Is re-flashing the only solution here? Is there a way to access the extlinux.conf file and make the change?

Hi,
I have observed the same modification of extlinux.conf after update, without problem in my case: there
It happens with update to JetPack 4.4 - L4T R32.4.3 production release
New cboot version doesn’t add “root=…” parameter (and other things from cbo.dtb) from CBO partition to kernel command string, so while bootloader updates extlinux.conf file (/boot/extlinux/extlinux.conf) is parsed to add this parameters with APPEND. In your case something go wrong and parameter is missing.
How to edit extlinux.conf from initramfs, where Xavier is stuck in case of error is a good question :(.

Hi,

The recovery instructions was posted to annocement thread. Please refer here:

Also please note that the issue is now fixed and we have reenabled the apt server for upgrading JetPack using debian package management tools.

1 Like

In my case, I reinstalled one of them, and did a “dd” dump from the working eMMC to an USB disk (not perfect as I did it while running on it, but working anyway). It can be used to boot another Jetson AGX by USB port (rectangular one, which is both USB 3 and eSATA)

https://sorel.pixconfig.fr/Manual/Working-Jetson-AGX-eMMC-dump.img.gz

While the USB disk will be seen as /dev/sda, this image contains the working extlinux.conf, containing root=/dev/mmcblk0p1. So if you boot it, it will boot your root file system. So that you can correct your extlinux file, and simply reboot without the USB disk.

Following suhash link, another way seems to be :

Another workaround if you see the bash (initrd) console in UART after running OTA upgrade. 1. reboot the device 7 times, 2. the device should be able to boot up properly. 3. edit the /boot/extlinux/extlinux.conf to do following changes: - APPEND {cbootargs} quiet + APPEND {cbootargs} root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyTCU0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 4. Reboot the device, and run “dpkg-reconfigure nvidia-l4t-bootloader” 5. reboot device.

Anyway, I guess having simple USB or SDcard bootable images for AGX Xavier from NVIDIA would be lovely