USB headless model can't work any more on Jetson AGX Orin

[leon.richard.tan]

Two days ago, I first initialed jetson agx orin by USB headless model. Then I login by SSH and followed the instructions to install jetpack 5.0.2.

However, today, USB headless mode can’t work any more.

I connected my macOS computer to the Jetson AGX Orin Developer Kit’s USB Type-C port (10) next to the 40-pin connector, and couldn’t find the usb device on my macbook.

% ls /dev/cu.usbmodem*
zsh: no matches found: /dev/cu.usbmodem*

Then I checked usb service on jetson agx orin, the service is running.

leon@jetsonAGXOrin:~$ systemctl status nv-l4t-usb-device-mode.service
● nv-l4t-usb-device-mode.service - Configure USB flashing port for device mode
     Loaded: loaded (/etc/systemd/system/nv-l4t-usb-device-mode.service; enabled; vendor preset: enabled)
     Active: active (exited) since Sun 2023-01-01 15:50:08 +08; 5min ago
    Process: 434 ExecStart=/opt/nvidia/l4t-usb-device-mode/nv-l4t-usb-device-mode-start.sh (code=exited, status=0/SUCCESS)
   Main PID: 434 (code=exited, status=0/SUCCESS)

Jan 01 15:50:07 jetsonAGXOrin systemd[1]: Starting Configure USB flashing port for device mode...
Jan 01 15:50:08 jetsonAGXOrin nv-l4t-usb-device-mode-start.sh[434]: /
Jan 01 15:50:08 jetsonAGXOrin nv-l4t-usb-device-mode-start.sh[900]: /sys/devices/platform/3520000.xusb_padctl/usb2-0/usb_role/usb2-0-role-switch
Jan 01 15:50:08 jetsonAGXOrin systemd[1]: Finished Configure USB flashing port for device mode.

I changed to my another macbook, I changed the usb cable, I unplug DP cable, it can’t work.

By the way, I try to enter Force Recovery Mode, but failed.

And I disconnected jetson agx orin from my host macbook, and connected my old jetson nano, the USB headless mode worked.

~ % ls /dev/cu.usbmodem* 
/dev/cu.usbmodem14230191045563
~ % ssh leon@192.168.55.1
leon@192.168.55.1's password: 

Then I suspected the USB Type-C port (10) was broken. I connected my USB flash disk to port (10), Ubuntu on jetson agx orin detected nothing. After I connected my USB flash disk to USB Type-A port (12), the disk was mounted on Ubuntu.

Could I conclude that USB Type-C port (10) was broken?

My hosts are macbook air m2 and macbook 16’'.

[DaneLLL]

Hi,
Please check and see if you can try this:
Jetson Orin Dev Kit SSH over USB - #3 by DaneLLL

See if you can connect Orin developer kit to a Linux host and try to login through ssh command.

[leon.richard.tan]

Thank you for your reply. But the USB headless mode can’t work. My host macbook can’t detect usb modem from Jetson AGX Orin.

% ls /dev/cu.usbmodem*
zsh: no matches found: /dev/cu.usbmodem*

But my host macbook did detect usb modem from my old Jetson Nano, and login through 192.168.55.1.

~ % ls /dev/cu.usbmodem* 
/dev/cu.usbmodem14230191045563
~ % ssh leon@192.168.55.1
leon@192.168.55.1's password: 

And my USB flash disk can’t mount to Ubuntu on Jetson AGX Orin through USB port(10). So I suspect this USB port(10) is broken. Could you help me to use your flash disk to try port(10) in your Orin? If your port(10) works, but my port(10) can not, I can conclude that my port(10) is broken.

Thanks!

[linuxdev]

@leon.richard.tan

My host macbook can’t detect usb modem from Jetson AGX Orin.

% ls /dev/cu.usbmodem*
zsh: no matches found: /dev/cu.usbmodem*

It might be relevant to know that on Linux USB serial devices have different naming conventions depending on the driver for the UART (e.g., an FTDI UART has a different name compared to other manufacturers because they use different drivers, and it is a device special file created by the driver). Are you sure that the UART you are looking at uses that naming convention? I don’t have a Mac, but if there is a log similar to the Linux “dmesg”, then perhaps you could see what appears in the log upon plugin of the device. It might be there, but using a different name (and different releases of Mac o/s might use different drivers). A log would also answer the question about whether the USB port is broken should other USB ports have a log message, but that particular one lacks a log message.

[leon.richard.tan]

When I first unboxed the Orin, login through headless mode by sudo screen /dev/cu.usbmodemxxxxxxxxx 115200, and initiate the ubuntu, the USB port(10) worked. At that time, I didn’t have a DP to HDMI cable, so I could only use headless mode to make my host login my Orin.

When the ubuntu initiated, I login my Orin through wifi by ssh leon@192.168.68.116.

However, several days later, when I wanted to use Jetson SDK Manager, I found the USB port(10) was out of work.

[leon.richard.tan]

[linuxdev]

I can’t see the video, it claims my Firefox browser does not support the CODEC (the forum is saying that, not my browser).

Does your Mac have any kind of log system which would tell you each time it sees something plugged in to USB, along with details? If this were Linux, then you could monitor “dmesg --follow”, and then look for new logs as a device is plugged in. Do you have any other Linux computer you could use just to plug in the USB and see the logs? This would at least verify issues following the host versus following the Jetson.

[leon.richard.tan]

I used Nano as a host to connect the port(10) on Orin.

leon@leon-jetson-nano:~$ dmesg | grep --color 'tty'
[    0.000000] Kernel command line: tegraid=21.1.2.0.0 ddr_die=4096M@2048M section=512M memtype=0 vpr_resize usb_port_owner_info=0 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 console=ttyS0,115200n8 debug_uartport=lsport,4 earlyprintk=uart8250-32bit,0x70006000 maxcpus=4 usbcore.old_scheme_first=1 lp0_vec=0x1000@0xff780000 core_edp_mv=1075 core_edp_ma=4000 gpt tegra_fbmem=0x800000@0x92ca9000 is_hdmi_initialised=1  earlycon=uart8250,mmio32,0x70006000  root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 quiet root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 
[    0.001734] console [tty0] enabled
[    1.120043] console [ttyS0] disabled
[    1.120177] 70006000.serial: ttyS0 at MMIO 0x70006000 (irq = 63, base_baud = 25500000) is a Tegra
[    1.120356] console [ttyS0] enabled
[    1.121683] 70006040.serial: ttyTHS1 at MMIO 0x70006040 (irq = 64, base_baud = 0) is a TEGRA_UART
[    1.122259] 70006200.serial: ttyTHS2 at MMIO 0x70006200 (irq = 65, base_baud = 0) is a TEGRA_UART
leon@leon-jetson-nano:~$ ls -l /dev/ttyACM0
ls: cannot access '/dev/ttyACM0': No such file or directory

And I also followed your suggestion, while got nothing.

leon@leon-jetson-nano:~$ dmesg --follow
...
[   68.300537] Extcon DP: HPD disabled
[   68.300568] hpd: hpd_switch 0
[   68.300605] hpd: switching from state 0 (Reset) to state 1 (Check Plug)
[   68.300669] hpd: state 1 (Check Plug), hpd 0, pending_hpd_evt 0
[   68.300723] hpd: switching from state 1 (Check Plug) to state 3 (Disabled)
[   68.311917] tegradc tegradc.0: unblank
[   68.311953] tegradc tegradc.1: blank - powerdown
^C

So, I really thought that the port(10) was broken.

[linuxdev]

In the first log excerpt, was this what resulted from the USB plugin? Incidentally, you can’t look for just “ttyACM0”, you really need to look for all “ttyACM*”. The number at the end is not always 0. USB increments this if the previous number is in use, and if there is an unplug followed by a plugin, then there might be a numeric increment despite the previous number disappearing only seconds later.

That looks like the correct port for serial console. The Nano should have seen a USB plugin. The last thing to check is whether the cable itself is failing. Is this the cable supplied by NVIDIA? If so, then this would be a good cable and suggests the port did indeed fail. If this is a “charger” cable, then about 2/3 of them fail due to lack of quality in the data wiring, and you’d need to check other cables.

[leon.richard.tan]

Thank you for your reply.
I have tried ls -l /dev/ttyA* and got nothing.
The USB cable is from Jetson AGX Orin SDK. The cable is ok. I have checked it by using it connect flash disk to macbook. And I changed to other cables.

And the port(10) couldn’t read flash disk while I connected flash disk to Orin through port(10). However, the flash disk can be read by Orin through other USB port, such as port(12).

So, I think the port(10) is indeed broken.

[linuxdev]

Sorry for the long response, there is a lot which can be confused when looking at the Orin’s micro-OTG port versus the type-C port the picture shows in use. I have an Orin which I cannot verify serial console on.

I don’t think the micro-OTG port has host mode available (I could be wrong though since I have not tried…my cables are all micro-B, and to use the port as a host a micro-A cable would be required…micro-B cables are not capable of triggering host mode behavior due to the ID selector pin). Maybe that port is ok for host mode if a micro-A cable is used, but I wouldn’t bet on it.

The use of an outside USB serial UART alters things a lot (the above is in regard to the integrated micro-OTG port). Certainly on most Jetsons the micro-OTG port is used only in device mode, but if the USB end of a USB serial UART is plugged in to a full-sized type-A connector (the device itself has a full-sized type-B connector), then it should work. A plugin event should register in logs if something connects to a host port (a type-B plug into a type-A socket).

During a flash the micro-OTG on earlier models is what is used for flash, which means the Jetson becomes a custom USB device. Orins (and Xaviers with the type-C port) use the type-C for device mode flash (the micro-OTG in that case is device mode for serial console, but isn’t actually needed for flash).

In one of your above pictures the type-C port of the AGX Orin (it’s a bit larger than a micro-OTG and is really type-C) is connected to the full-sized type-A port of the Nano. A type-C cable is a bit of an illusion; it does not matter which end plugs in, it should work for either device mode or host mode, but this is only because of extra wiring and detecting of host versus device. The type-A of the Nano implies the type-C on the Orin can only work with the Orin in device mode (such as for a recovery mode Orin). However, if plugin of the cable to the Nano fails to show logs upon plug of the recovery mode Orin, then you have solid evidence of the Orin failing, at least for virtual network device, but not for serial console. Despite failing, this is unlikely a hardware issue; virtual ethernet is a software service, and without that running, there won’t be any virtual ethernet on the USB-C.

Serial console is type micro-OTG port (in combination with a micro-B plug) on the other side of the Jetson from the USB-C you have connected. Can we verify if a micro-B cable plugged in to the micro-OTG port fails for serial console?

Incidentally, I have an Orin which does not appear to have the serial port showing up on the host PC, but I have errors in the “/opt/nvidia/l4t-usb-device-mode/” scripts (which I believe is a software issue).

[leon.richard.tan]

I have done a simple test on my AGX Orin.

Running dmesg --follow on Orin, then

1. connect a flash disk through port(10), while get nothing on terminal.
   

   

2. connect the flash disk through port(12), which is a USB type-A port, then dmesg --follow show some new message on terminal.
   

   
   
   
   

So I thought the port(10) was indeed broken.

By the way, when I received AGX Orin 3 weeks ago, the port(10) worked well, through which my macbook connected Orin in headless mode and setup Ubuntu successfully. But several days later, the port(10) couldn’t work.

[linuxdev]

It might actually be broken (I can’t say for certain). It could still be software though, e.g., an update might have changed something. I don’t know what check would be needed to verify things like device tree.

[leon.richard.tan]

Thanks any way! :-)