After attempting to update to Jetpack 5.02, the Jetson Xavier NX won’t boot and Linux host fails to communicate via usb with the Jetson Xavier NX so we can’t flash. dmesg --follow not showing the relevant usb device and we could not put the Jetson into force recovery mode either ( lsusb never shows “nvidia” on any bus line ). The host has the sdkmanager ready (files already downloaded successfully) but shows “could not detect board” in red and won’t refresh (in step 2 of HARDWARE CONFIGURATION). we checked the usb cable and its seems to work fine. Based on what we found online so far we suspect an issue with nv4-l4t-usb-device-mode.service apparently not available and not working.
When I run: sudo systemctl enable nv-l4t-usb-device-mode.service it yields: Failed to enable unit: Unit file nv-l4t-usb-device-mode.service does not exist.
We checked /opt/nvidia and it doesn’t have l4t-usb-device-mode directory and no nv-l4t-usb-device-mode-start.sh or l4t-usb-device-mode-stop.sh anywhere. We can’t find any docs on how to configure and start the specific nv4-l4t-usb-device-mode.service which we think is what’s missing.
Another point here is that the Jetson Xavier NX version we have is eMMC and not the micro-SD card slot, so the question here is are we missing USB drivers support for Jetson Xavier NX with eMMC? if so, how do we get this support? What do we do next? any idea?
The device mode service is something that runs on the Jetson itself when fully booted. Were your commands for systemctl run on the Jetson? If so, then it indicates the Jetson fully booted. When in recovery mode a Jetson is not booted, and that particular device mode service is unrelated, but the NX would appear to be a custom device with some similar lsusb output on the host PC if the micro-B USB is connected from recovery mode Jetson to the host PC. Can you verify:
Which USB was used to connect between the NX and host PC?
If the recovery pins were shorted followed by applying power and disconnecting recovery pins, or else shorting recovery pins and resetting power and then removing recovery pins?
If the Jetson is in recovery mode, and on the host PC you monitor “dmesg --follow”, what log lines show up when you plug in the USB between the Jetson’s micro-OTG USB and the host PC?
Note that if you saw the log lines in your screenshot, then it means the Jetson booted. It might not have GUI working, but otherwise it is probably fully functional. The next step would be to post a full serial console boot log rather than a screenshot. A serial console boot log shows a lot. The Nano and NX instructions for serial console are basically the same (post this if boot fails): https://jetsonhacks.com/2019/04/19/jetson-nano-serial-console/
Systemctl was not run on Jetson, only on the host. It appears we have no access to shell on the Jetson at this point.
The USB is a standard port on a Dell Inspiron 16 Plus (we tried both USB-C and USB3 ports)
yes the recovery pins were shorten witha jumper between 9 and 10 followed by applied power and disconnecting the pins. For the other option, not sure how you’d of resetting the power. You mean CTRL-ALT-DELETE?
No new log lines appear on dmesg --follow when we plug in the USB. There is no change at all in dmesg or lsusb whether we plug-in or out
the log lines in the screenshot are from the Jetson not in recovery! there is no way that we see to access the Jetson so we can’t submit a log file. That’s why the screenshot
Alright. I think what you are suggesting is that we will connect a serial monitor and submit a log file if the boot fails. (Based on the video instructions you shared we will need to purchase the cable) - I suppose we can try this next. Anything else you could tell us based on what we see so far? (normal boot appears to fail) does it look like a hardware issue or software issue?
We have attempted to connect to a serial monitor as you advised . The output log of dmesg --follow on the host is this:
[15698.675531] usb 3-6: new full-speed USB device number 23 using xhci_hcd
[15698.871564] usb 3-6: device descriptor read/64, error -71
[15699.175579] usb 3-6: device descriptor read/64, error -71
[15699.479543] usb 3-6: new full-speed USB device number 24 using xhci_hcd
[15699.675583] usb 3-6: device descriptor read/64, error -71
[15699.979588] usb 3-6: device descriptor read/64, error -71
[15700.087704] usb usb3-port6: attempt power cycle
[15700.567563] usb 3-6: new full-speed USB device number 25 using xhci_hcd
[15700.567798] usb 3-6: Device not responding to setup address.
[15700.775795] usb 3-6: Device not responding to setup address.
[15700.983571] usb 3-6: device not accepting address 25, error -71
[15701.387569] usb 3-6: new full-speed USB device number 26 using xhci_hcd
[15701.387795] usb 3-6: Device not responding to setup address.
[15701.595815] usb 3-6: Device not responding to setup address.
[15701.803584] usb 3-6: device not accepting address 26, error -71
[15701.803726] usb usb3-port6: unable to enumerate USB device
(got similar -71 error in trying on two different host linux Ubuntu 18.4 laptops) we can’t connect minicom… (please not that we tried different ports and all give same errors. But if we plug-in another device, such as a bluetooth mouse they work). If we are not mistaken boot fails even using USB - Serial . Anything else we could try? What would you recommend next?
Yes, that or a hardware based power reset. Any reset. But since you had the recovery pins active during power up, this won’t matter. Your usage should have correctly placed the Jetson in recovery mode.
The USB port on the host PC is not necessarily what I’m wondering about. Which port did you use on the Jetson? If it was a micro-OTG port, then did you use a supplied dev kit cable? Most any host PC port will work, but only one port on the Jetson works (and ports differ depending on model). In the case of a micro-OTG port the micro-B USB cable (often called a “charger” cable) is quite often an issue if it isn’t the one NVIDIA provides. In cases where it is a type-C port (some models use this), then cable quality is rarely an issue.
If the Jetson is in host mode, then there would be no message on “dmesg --follow”. If the Jetson is in device mode (which is the case during recovery mode), then the lack of message indicates a problem. This is why I wanted to make sure it was actually in recovery mode. Not all Jetson ports are active during recovery mode, and thus I am hoping to verify the correct Jetson port is used (this depends on model). Should this be the correct port with a recovery mode Jetson, then I’d say try plugin to a different host PC running “dmesg --follow” if you can; if this fails there too, then you probably have a hardware failure.
Serial console logs work without a monitor. A separate PC is used as a terminal, and that other PC has no requirement for the Jetson to have a monitor. Graphics (and most parts) of the Jetson could crash and burn, and serial console would still work. Serial console even works during bootloader stages before Linux ever starts (which is very valuable debug information). Have you tried serial console? Is serial console also failing?
Honestly, that serial console is one of the simplest and most valuable tools a developer can have. Life will be easier if you have that, so I would suggest that is the next step. The fact that the monitor does have some output says the Jetson is not actually dead. The full serial console boot log could say a lot. There is very little else left to try (perhaps swapping the module to a different carrier board would help, but that’s more extreme than a serial console log).
I agree about the incredible usefulness of the serial console. I have used it more than once on other projects (such as Arduino) and I can’t imagine working without it. (I custom-made one after you suggested)
It turns out that the micro-B USB cable was the culprit! what you said about the Nvidia cable (which for some reason wasn’t provided with the purchase) made me check the initial cable I used (bad) and I was able to find another cable that worked! as soon as it connected, the NX to the host dmesg --follow (or lsusb) showed it is communicating as expected.
The Jetson Xavier NX is now flashed, and re-installed with JetPack 5.02. (just a hunch the source for the initial breakdown during JetPack update was perhaps running out of space on the Xavier NX (?) once I expanded the memory to a 256 Micro SD, it seems to function well.
Thank you so much for your help and a Happy New Year!
I’ve dissected a few charger cables before, and it is astonishing the number of which have had either a single strand of thin copper or two strands for the data. So it never surprises me to see a “charger” cable (almost all of them are sold as such for people charging cell phones) fail data.
I don’t know if there are some models of Jetson which have the micro-OTG port and don’t come with the micro-B USB cable. All of the dev kits with that kind of port which I’ve seen do come with that cable.
Running out of space would still leave a visible boot, but login might not work. During flash no extra applications are actually installed, but at the end of flash, the Jetson automatically reboots, and once there is an account set up, that is when the optional applications are installed which might overfill the drive. If you had not completed first boot account setup, then it wouldn’t be possible for lack of space (on the Jetson side) causing the problem (you’d know if you had got to the point of completing first boot account setup; if not, then there was no extra content install).
