USB Devices not working

Jetson NX used to boot fine and had USB devices working. And now none of the USB ports or devices are working.

lsusb generates the following with additional USB peripherals being attached:

$ lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Looked at dmesg and get the following:

$ dmesg | grep -i USB
[    0.000000] Kernel command line: console=ttyTCU0,115200 video=tegrafb no_console_suspend=1 earlycon=tegra_comb_uart,mmio32,0x0c168000 gpt tegra_fbmem=0x800000@0xa069d000 lut_mem=0x2008@0xa0697000 usbcore.old_scheme_first=1 tegraid=19.1.2.0.0 maxcpus=6 boot.slot_suffix= boot.ratchetvalues=0.4.2 vpr_resize sdhci_tegra.en_boot_part_access=1    quiet root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyTCU0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0
[    0.841121] usbcore: registered new interface driver usbfs
[    0.841169] usbcore: registered new interface driver hub
[    0.841258] usbcore: registered new device driver usb
[    0.903568] vdd-usb-3v3: at 3300 mV
[    1.846328] usbcore: registered new interface driver r8152
[    1.846400] usbcore: registered new interface driver asix
[    1.846487] usbcore: registered new interface driver ax88179_178a
[    1.846527] usbcore: registered new interface driver cdc_ether
[    1.846573] usbcore: registered new interface driver net1080
[    1.846611] usbcore: registered new interface driver cdc_subset
[    1.846645] usbcore: registered new interface driver zaurus
[    1.846706] usbcore: registered new interface driver cdc_ncm
[    1.846929] usbcore: registered new interface driver qmi_wwan
[    1.850442] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    1.850635] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    1.858072] tegra-xusb 3610000.xhci: USB2 port 0 has OTG_CAP
[    1.860041] tegra-xusb 3610000.xhci: extcon 0: ffffffc1f5233000 id
[    1.860648] usbcore: registered new interface driver cdc_wdm
[    1.860683] usbcore: registered new interface driver uas
[    1.860735] usbcore: registered new interface driver usb-storage
[    1.860811] usbcore: registered new interface driver usbserial
[    1.860844] usbcore: registered new interface driver option
[    1.860894] usbserial: USB Serial support registered for GSM modem (1-port)
[    1.862203] tegra-xusb 3610000.xhci: Firmware timestamp: 2019-07-24 05:47:34 UTC, Version: 60.06 release
[    1.862234] tegra-xusb 3610000.xhci: xHCI Host Controller
[    1.862251] tegra-xusb 3610000.xhci: new USB bus registered, assigned bus number 1
[    1.862966] tegra-xusb 3610000.xhci: hcc params 0x0184ff25 hci version 0x110 quirks 0x00050810
[    1.863006] tegra-xusb 3610000.xhci: irq 463, io mem 0x03610000
[    1.863401] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[    1.863407] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    1.863411] usb usb1: Product: xHCI Host Controller
[    1.863416] usb usb1: Manufacturer: Linux 4.9.140-tegra xhci-hcd
[    1.863420] usb usb1: SerialNumber: 3610000.xhci
[    1.864055] hub 1-0:1.0: USB hub found
[    1.864602] tegra-xusb 3610000.xhci: xHCI Host Controller
[    1.864612] tegra-xusb 3610000.xhci: new USB bus registered, assigned bus number 2
[    1.864619] tegra-xusb 3610000.xhci: Host supports USB 3.1 Enhanced SuperSpeed
[    1.865371] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003
[    1.865376] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    1.865381] usb usb2: Product: xHCI Host Controller
[    1.865386] usb usb2: Manufacturer: Linux 4.9.140-tegra xhci-hcd
[    1.865390] usb usb2: SerialNumber: 3610000.xhci
[    1.865861] hub 2-0:1.0: USB hub found
[    1.867364] usbcore: registered new interface driver xpad
[    1.966469] usb usb1: usb_suspend_both: status 0
[    2.102194] usbcore: registered new interface driver usbhid
[    2.102198] usbhid: USB HID core driver
[    2.117001] usbcore: registered new interface driver snd-usb-audio
[    6.033989] tegra-xusb 3610000.xhci: Upgrade port 0 to USB3.0
[    6.033995] tegra-xusb 3610000.xhci: Upgrade port 1 to USB3.0
[    6.134503] usb usb2: usb_suspend_both: status 0
[    6.367058] usb0: HOST MAC 36:d1:a3:90:6a:9a
[    6.367098] usb0: MAC 36:d1:a3:90:6a:9b
[    6.403929] l4tbr0: port 2(usb0) entered blocking state
[    6.403938] l4tbr0: port 2(usb0) entered disabled state
[    6.404138] device usb0 entered promiscuous mode
[    6.409298] IPv6: ADDRCONF(NETDEV_UP): usb0: link is not ready
[    8.278197] tegra-xusb 3610000.xhci: entering ELPG
[    8.281742] tegra-xusb 3610000.xhci: entering ELPG done

If this is caused by the device tree, what could change the device tree setting?

TIA.

Hi,
Do you use Xavier NX developer kit? If it worked and then fails, probably hardware is broken. We would suggest do re-flash and check if it works. If not, may need to do RMA:
Jetson FAQ | NVIDIA Developer

@DaneLLL - I thought it was the hardware too so I used my spare Aetina AN810. The USB does not work on either of the AN810. I can SSH into the Jetson, but no USB devices work. I had installed JetPack 4.4 and it worked till now somehow USB got busted. I find it really odd that USB isnt working??
/a

A device tree problem could in theory make USB fail, and this could be related to some sort of update or mismatch in software. However, that’s very unlikely under the circumstances. The reason you should try flashing (and you could use a different SD card), is that it will guarantee that the image and other software within the NX have a matching version. If the SD still fails, then it is extremely likely it is hardware failure.

I am using Aetina AN810 board and Production Module (SOM) so there is no SD Card. After flashing it, we usually move it to use NVME to boot.
I will try to reflash and see if it works.
/a

…ahh, the above is quite important! This implies the device tree must be quite different than what you’d get from the generic NVIDIA dev kit flash software. The hardware might be ok after all!

When you are using a third party carrier board you must use the board support package they provide. The device tree (and possibly kernel) cannot be predicted by NVIDIA, and thus the company producing the board will provide their own customized version of what NVIDIA supplies. The third party flash software will just be a customization of what is published by NVIDIA, but without this, I’d expect some or all of USB to fail.

I usually flash the SOM with the JetPack4.4.1 using the NVIDIA SDK as some applications are not supported in the latest jetpack. I will test it default NVIDIA JetPack to make sure everything is working before I apply the vendors patch and let you know how it works out.
Thanks
/a

If this is a third party carrier board, then using JetPack will fail to work correctly except for those parts of the custom carrier board which are an exact match to the dev kit carrier board. Does the manufacturer of your carrier board provide a board support package for flashing? Use that instead.

@linuxdev - when I purchase the SOM, I just flash it with JetPack 4.4.1 to make sure all is good. Then I usually reflash with the vendor’s patch. Usually the default jetpack will let me configure the make sure the USB etc is all working. Prsently I cannot even reflash it will NVIDIA JetPack 4.4.1 with our without vendor’s patch.
NVIDIA SDK gives the following error:

1:09:26 INFO: Flash Jetson Xavier NX - flash: [ 3.9061 ] tegrarcm_v2 --oem platformdetails eeprom cvm /home/aidev/nvidia/nvidia_sdk/JetPack_4.4.1_Linux_JETSON_XAVIER_NX/Linux_for_Tegra/bootloader/cvm.bin

21:09:26 INFO: Flash Jetson Xavier NX - flash: [ 3.9080 ] MB2 Applet version 01.00.0000

21:09:26 INFO: Flash Jetson Xavier NX - flash: [ 3.9570 ] tegrarcm_v2 --ismb2

21:09:26 INFO: Flash Jetson Xavier NX - flash: [ 3.9662 ] tegrarcm_v2 --reboot recovery

21:09:26 INFO: Flash Jetson Xavier NX - flash: [ 3.9680 ] MB2 Applet version 01.00.0000

21:09:28 INFO: Flash Jetson Xavier NX - flash: [ 3.9849 ]

21:09:29 INFO: Flash Jetson Xavier NX - flash: Parsing board information failed.

21:09:29 ERROR: Flash Jetson Xavier NX - flash: command terminated with error

**21:09:29 SUMMARY: Flash Jetson Xavier NX - flash: First Error: Installation failed.**

I know I ’ in recovery mode as I can see the UART output confirming that:

[0565.966] I> Welcome to MB2(TBoot-BPMP) Applet (version: 00.00.2018.32-mobile-)
[0565.974] W> Profiler not initialized                                          
[0565.977] I> DMA Heap @ [0x40020000 - 0x40065800]                              
[0565.982] I> Default Heap @ [0xd486400 - 0xd48a400]                            
[0565.987] W> Profiler not initialized                                          
[0565.990] W> Profiler not initialized                                          
[0565.994] E> DEVICE_PROD: Invalid value data = 0, size = 0.                    
[0565.999] W> device prod register failed                                       
[0566.003] W> Profiler not initialized                                          
[0566.064] I> sdmmc DDR50 mode                                                  
[0566.068] I> QSPI Flash Size = 32 MB                                           
[0566.079] I> Qspi initialized successfully                                     
[0566.130] E> Link startup dme_set failed                                       
[0566.134] E> UFS initialization failed                                         
[0566.138] I> UFS is not present                                                
[0566.141] W> Profiler not initialized                                          
[0566.147] W> Cannot find any partition table for 00000003                      
[0566.155] I> Found 11 partitions in SDMMC_USER (instance 3)                    
[0566.163] I> Found 47 partitions in QSPI_FLASH (instance 0)                    
[0566.168] W> Profiler not initialized                                          
[0566.172] W> Profiler not initialized                                          
[0566.175] W> Profiler not initialized                                          
[0566.179] I> Entering 3p server                                                
[0566.182] I> USB configuration success                                         
[0566.992] I> Populate eeprom info                                              
[0566.997] I> Populate eeprom info for module cvm                               
[0567.054] I> Rebooting : reboot-recovery

I also check with with lsusb and can see NVIDIA Corp in the list:
Bus 002 Device 032: ID 0955:7e19 NVidia Corp.

Is this SOM any good? I believe it is still under warranty and if you can confirm if it is for RMA I can open an RMA for it.

Thanks.

/a

I can’t answer what the third party carrier board does differently. This is something you should directly ask the manufacturer about. However, I can give some “maybe” advice.

When in recovery mode the Jetsons tend to not care about third party carrier boards versus dev kit boards. Technically, I think “most” Jetsons could be flashed with either third party or dev kit flash software when mounted to any carrier board, but most debugging to know about a true recovery mode failure requires the dev kit carrier board.

The eMMC version of module will differ versus the SD card model. I don’t know if you could expect the eMMC model to function (fully) on the dev kit carrier board, but probably it would work if given the correct device tree. Almost certainly there would be issues in using the eMMC module with the wrong device tree on any of the carrier boards, and USB would be a prime candidate for “wrong device tree”.

Do you have a dev kit carrier board you can test flash on? I would say that if the module has problems flashing with the dev kit software on the dev kit carrier board, then it might be time to RMA (though it would also require seeing flash logs in case it is something simple going on in software and not a hardware issue). Certainly it is possible that the carrier board itself is at issue, and if it is a third party carrier board, then the module would be ok and you’d need a new carrier board.

You are in a tough situation because I think you need a second (dev kit) carrier board to really confirm. Or you need the company with that carrier board to confirm. Without the right combination of carrier board and module to test with there will always be some guessing.

Do note that seeing the Jetson with lsusb while in recovery mode is a sign the module is probably good, and perhaps at least part of the carrier board is also good.

I believe have the NX Devkit with me. I will add the board to that devkit and try to re-flash it.
Thanks,
/a