Bricked Xavier AGX

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier
,
1

i found an old Xavier AGX (bought around 2020) when I moved recently and decided (wrong-headed-ly) to UPGRADE it… I did the usual ubuntu do-release-upgrade with all the apt upgrading and got the device up to Ubu 22.04. Of course Jetpack 6 doesn’t run on this combo and I understand 22.04 is not supported on Xavier AGX.

SDKManager doesn’t find the device (seems like the USB is not connecting or enumerating).

Is there any way to ‘downgrade’ this device back to L4T 35 without SDKManager? Can I put an image on an SD card? I can SSH to the device… is there something I can download on the device and downgrade?

FWIW - Yes, I did try to force the update with the buttons and with the shutdown --force update command all to no avail.

3

Did you put in force recovery mode correctly?

NO.
SDK Manager is the only way.

4

Incidentally, it isn’t bricked. “Bricked” implies the software or hardware is permanently damaged and the only fix is by unsoldering components, reprogramming, and then soldering components back in.

Jetsons do not have an actual BIOS. That means any software which tries to add boot content without understanding this will cause boot to fail. This is why you can’t use the stock Ubuntu 22.04. You’ll be limited to 20.04. This is also why you can’t just load a new o/s onto an SD card of an eMMC model (the BIOS isn’t there to see the content).

I realize this won’t be a “happy” thing, but you’d have to upgrade to Orin to start working with mainline kernel. This is when the boot chain becomes fully UEFI compatible. The UEFI has a hardware abstraction layer which hides the custom boot methods and allows a generic boot loader to work without special knowledge (there might still be some hoops to jump through for many situations).

This is the release compatibility chart, and Xavier only goes to L4T R35.x (which is Ubuntu 20.04):
https://developer.nvidia.com/linux-tegra

Also, in recovery mode a Jetson is a custom USB device understood only by a custom driver that runs in Linux. This is why you need a Linux host PC to flash.

5

Thank you both for taking the time to respond, but perhaps my query was unclear.

Yes, i understand the device is not an absolute brick. I also understand that 22.04 is not compatible and I’m not trying to make it so. I simply wish to roll back the install to a compatible L4T version.

Yes,. i now that force recovery mode is the recommended way, and i attempted several times as described in my post. Yet, the host would not recognize the usb device and yes i plugged direct usb c to c from the connector on the front of the AGX (not above the power pin) to my dev host.

Given that the device does connect to my network and that I can SSH to the device, is there no way to ‘sideload’ L4T 35.5 and unpack it directly onto the eMMC?

6

to be more clear - the device will not enter forced recovery mode. I suspect I blew away the relevant drivers, etc. for this. Is there a way to re-install them? Or is there a way to install L4T directly on the device – NOT over USB.

7

NO.
Force recovery mode has nothing to do with software drivers.
The board is physically broken if it cannot even enter force recovery mode.

8

Recovery mode turns the Jetson into a custom USB device that uses a custom driver (which is part of the flash software). Recovery mode in and of itself does not alter a Jetson until the USB driver does so. Recovery mode is actually quite reliable and rarely does the hardware fail. Most of the time the issue is someone using a VM that is not set up to correctly pass the USB through. Sometimes it is a cable issue, but this tends to only be on modules that use a micro-OTG USB connector (USB-C is very reliable).

For clarity, consider the “recovery” button (or pins on some models) to be like the shift key of a keyboard when we want to create a capital letter…we hold the shift key, tap the key we wish to capitalize, and then let go of the shift key. The recovery button modifies either the power on event or the power reset event. If the recovery is held down during either power on or power reset, then it starts in recovery mode.

I suggest on your Linux host PC that you monitor logs with “dmesg --follow”. Without the USB connected, I suggest you then start the Jetson in recovery mode. At this point, if you monitor the dmesg logs, does anything show up when you plug in the USB cable? If anything shows up, then it is alive.

9

thanks again for taking the time to follow up. I’ve been doing exactly as you described – monitoring the host USB waiting for the device. I did see it flash very briefly once, but when I checked with lsusb the device was gone.

If I subsequently simply restart the device (push the reset button without the force recovery), I DO see a brief, on-screen error saying that USB for flash recovery could not be started. When I check the logs - there is no trace of the error message though.

often these kind of boot mode pins can be tricky to get right. Is there a header or jumper I can set or some minimum/maximum time to hold the force_recovery pin?

Is the boot mode implemented in SOC firmware? could it have been erased? or does the force_recovery select a different kernel cmd line or something that I could have obliterated by my bad upgrading?

10

I think I observed the problem! See this snippet from the host’s syslog:

May 28 09:03:01 dev kernel: [757198.049223] usb 1-4: new high-speed USB device number 31 using xhci_hcd
May 28 09:03:01 dev kernel: [757198.197857] usb 1-4: New USB device found, idVendor=0955, idProduct=7019, bcdDevice= 1.02
May 28 09:03:01 dev kernel: [757198.197864] usb 1-4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
May 28 09:03:01 dev kernel: [757198.197866] usb 1-4: Product: APX
May 28 09:03:01 dev kernel: [757198.197868] usb 1-4: Manufacturer: NVIDIA Corp.
May 28 09:03:01 dev mtp-probe: checking bus 1, device 31: "/sys/devices/pci0000:00/0000:00:14.0/usb1/1-4"
May 28 09:03:01 dev mtp-probe: bus: 1, device: 31 was not an MTP device
May 28 09:03:01 dev mtp-probe: checking bus 1, device 31: "/sys/devices/pci0000:00/0000:00:14.0/usb1/1-4"
May 28 09:03:01 dev mtp-probe: bus: 1, device: 31 was not an MTP device
May 28 09:03:01 dev kernel: [757198.645046] usb 1-4: USB disconnect, device number 31

Any suggestions? fwiw - my host is Ubuntu 22.04 and I was planning on running SDKManager in a 20.04 VM.

11

I added udev rules as per the dev guide and to bypass the MTP business. However, the device will STILL not enumerate:

May 28 10:24:34 dev kernel: [ 2977.640941] usb 1-4: new high-speed USB device number 17 using xhci_hcd
May 28 10:24:34 dev kernel: [ 2977.789656] usb 1-4: New USB device found, idVendor=0955, idProduct=7019, bcdDevice= 1.02
May 28 10:24:34 dev kernel: [ 2977.789673] usb 1-4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
May 28 10:24:34 dev kernel: [ 2977.789681] usb 1-4: Product: APX
May 28 10:24:34 dev kernel: [ 2977.789685] usb 1-4: Manufacturer: NVIDIA Corp.
May 28 10:24:34 dev kernel: [ 2977.863460] usb 1-4: USB disconnect, device number 17
12

And the issue was the USB cable!

I was using a USB-C to USB-C cable that works for several other projects. But the only way I could get my host to see the device was to use a USB-A (host) to USB-C (device) cable.

FWIW, I was able to use the same C-C cable to ‘see’ the device on a Mac and a Chromebook, so I was able to verify that the device was entering recovery properly and that the issue was on the host side.

Thanks for staying with me on this and I hope the thread helps someone else in the future.

13

Glad you got it working. It is usually difficult to distinguish between a failing cable and a VM USB pass-through issue.

15

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.