USB C port "crashing"

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier
1

Recently had a USB port “crash” (Port 2-3 to be exact). Alternating to the other port it still worked. Here is the dmesg log showing a specific tegra-xusb error when moving two cameras between the usb ports.
[ 1324.459403] usb 1-1: USB disconnect, device number 3
[ 1324.478704] ucsi_ccg 1-0008: port0 evt: Type-C Port Disconnect Detected
[ 1324.480728] ucsi_ccg 1-0008: [typec-port0] Cable state:0, cable id:2
[ 1324.602325] usb usb1: usb_suspend_both: status 0
[ 1326.990379] ucsi_ccg 1-0008: port0 evt: Type-C Port Connect Detected
[ 1326.992212] ucsi_ccg 1-0008: [typec-port0] Cable state:1, cable id:2
[ 1332.053669] ucsi_ccg 1-0008: port1 evt: Type-C Port Disconnect Detected
[ 1332.055489] ucsi_ccg 1-0008: [typec-port1] Cable state:0, cable id:2
[ 1332.135929] usb 2-1: USB disconnect, device number 2
[ 1332.281242] usb usb2: usb_suspend_both: status 0
[ 1332.281257] tegra-xusb 3610000.xhci: entering ELPG
[ 1332.281276] tegra-xusb 3610000.xhci: 2-3 is not suspended: 0a000302
[ 1332.281281] tegra-xusb 3610000.xhci: entering ELPG failed
[ 1334.941961] ucsi_ccg 1-0008: port0 evt: Source Disabled State Entered
[ 1343.696363] ucsi_ccg 1-0008: port1 evt: Type-C Port Connect Detected
[ 1343.698183] ucsi_ccg 1-0008: [typec-port1] Cable state:1, cable id:2
[ 1347.597370] usb 2-1: new SuperSpeed USB device number 3 using tegra-xusb
[ 1347.618886] usb 2-1: New USB device found, idVendor=1e10, idProduct=4000
[ 1347.618912] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1347.618916] usb 2-1: Product: Blackfly S BFS-U3-50S5C
[ 1347.618920] usb 2-1: Manufacturer: FLIR
[ 1347.618924] usb 2-1: SerialNumber: 01277BB7
[ 1351.650276] ucsi_ccg 1-0008: port1 evt: Source Disabled State Entered
[ 1640.082944] ucsi_ccg 1-0008: port1 evt: Type-C Port Disconnect Detected
[ 1640.084762] ucsi_ccg 1-0008: [typec-port1] Cable state:0, cable id:2
[ 1640.141570] usb 2-1: USB disconnect, device number 3
[ 1640.285710] usb usb2: usb_suspend_both: status 0
[ 1640.285724] tegra-xusb 3610000.xhci: entering ELPG
[ 1640.285743] tegra-xusb 3610000.xhci: 2-3 is not suspended: 0a000302
[ 1640.285748] tegra-xusb 3610000.xhci: entering ELPG failed
[ 1642.985569] ucsi_ccg 1-0008: port1 evt: Type-C Port Connect Detected
[ 1642.987415] ucsi_ccg 1-0008: [typec-port1] Cable state:1, cable id:2
[ 1646.875107] usb 2-1: new SuperSpeed USB device number 4 using tegra-xusb
[ 1646.896669] usb 2-1: New USB device found, idVendor=1e10, idProduct=4000
[ 1646.896678] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1646.896682] usb 2-1: Product: Blackfly S BFS-U3-50S5C
[ 1646.896685] usb 2-1: Manufacturer: FLIR
[ 1646.896689] usb 2-1: SerialNumber: 01277BB7
[ 1650.933270] ucsi_ccg 1-0008: port1 evt: Source Disabled State Entered
[ 1723.589502] ucsi_ccg 1-0008: port1 evt: Type-C Port Disconnect Detected
[ 1723.591324] ucsi_ccg 1-0008: [typec-port1] Cable state:0, cable id:2
[ 1723.686078] usb 2-1: USB disconnect, device number 4
[ 1723.828812] usb usb2: usb_suspend_both: status 0
[ 1723.828828] tegra-xusb 3610000.xhci: entering ELPG
[ 1723.828864] tegra-xusb 3610000.xhci: 2-3 is not suspended: 0a000302
[ 1723.828868] tegra-xusb 3610000.xhci: entering ELPG failed
[ 1737.424004] ucsi_ccg 1-0008: port1 evt: Type-C Port Connect Detected
[ 1737.425830] ucsi_ccg 1-0008: [typec-port1] Cable state:1, cable id:2
[ 1741.320540] usb 2-1: new SuperSpeed USB device number 5 using tegra-xusb
[ 1741.342065] usb 2-1: New USB device found, idVendor=1e10, idProduct=4000
[ 1741.342072] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1741.342094] usb 2-1: Product: Blackfly S BFS-U3-50S5C
[ 1741.342098] usb 2-1: Manufacturer: FLIR
[ 1741.342101] usb 2-1: SerialNumber: 01277BB7
[ 1745.384278] ucsi_ccg 1-0008: port1 evt: Source Disabled State Entered
[ 1811.714757] ucsi_ccg 1-0008: port0 evt: Type-C Port Disconnect Detected
[ 1811.716779] ucsi_ccg 1-0008: [typec-port0] Cable state:0, cable id:2
[ 1814.209624] ucsi_ccg 1-0008: port1 evt: Type-C Port Disconnect Detected
[ 1814.210846] usb 2-1: USB disconnect, device number 5
[ 1814.211445] ucsi_ccg 1-0008: [typec-port1] Cable state:0, cable id:2
[ 1814.354874] usb usb2: usb_suspend_both: status 0
[ 1814.354888] tegra-xusb 3610000.xhci: entering ELPG
[ 1814.354905] tegra-xusb 3610000.xhci: 2-3 is not suspended: 0a000302
[ 1814.354909] tegra-xusb 3610000.xhci: entering ELPG failed
[ 1829.695666] ucsi_ccg 1-0008: port1 evt: Type-C Port Connect Detected
[ 1829.697542] ucsi_ccg 1-0008: [typec-port1] Cable state:1, cable id:2
[ 1833.610297] usb 2-1: new SuperSpeed USB device number 6 using tegra-xusb
[ 1833.631911] usb 2-1: New USB device found, idVendor=1e10, idProduct=4000
[ 1833.631918] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1833.631923] usb 2-1: Product: Blackfly S BFS-U3-50S5C
[ 1833.631926] usb 2-1: Manufacturer: FLIR
[ 1833.631929] usb 2-1: SerialNumber: 01277BB5
[ 1837.673408] ucsi_ccg 1-0008: port1 evt: Source Disabled State Entered
[ 1840.464740] ucsi_ccg 1-0008: port0 evt: Type-C Port Connect Detected
[ 1840.466622] ucsi_ccg 1-0008: [typec-port0] Cable state:1, cable id:2
[ 1848.438934] ucsi_ccg 1-0008: port0 evt: Source Disabled State Entered
[ 1880.342922] ucsi_ccg 1-0008: port0 evt: Type-C Port Disconnect Detected
[ 1880.344810] ucsi_ccg 1-0008: [typec-port0] Cable state:0, cable id:2
[ 1882.798204] ucsi_ccg 1-0008: port1 evt: Type-C Port Disconnect Detected
[ 1882.800093] ucsi_ccg 1-0008: [typec-port1] Cable state:0, cable id:2
[ 1882.878750] usb 2-1: USB disconnect, device number 6
[ 1883.024220] usb usb2: usb_suspend_both: status 0
[ 1883.024237] tegra-xusb 3610000.xhci: entering ELPG
[ 1883.024257] tegra-xusb 3610000.xhci: 2-3 is not suspended: 0a000302
[ 1883.024262] tegra-xusb 3610000.xhci: entering ELPG failed
[ 1890.384463] ucsi_ccg 1-0008: port1 evt: Type-C Port Connect Detected
[ 1890.386255] ucsi_ccg 1-0008: [typec-port1] Cable state:1, cable id:2
[ 1894.276909] usb 2-1: new SuperSpeed USB device number 7 using tegra-xusb
[ 1894.298556] usb 2-1: New USB device found, idVendor=1e10, idProduct=4000
[ 1894.298580] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1894.298586] usb 2-1: Product: Blackfly S BFS-U3-50S5C
[ 1894.298589] usb 2-1: Manufacturer: FLIR
[ 1894.298593] usb 2-1: SerialNumber: 01277BB7
[ 1898.369804] ucsi_ccg 1-0008: port1 evt: Source Disabled State Entered
[ 1900.777900] ucsi_ccg 1-0008: port0 evt: Type-C Port Connect Detected
[ 1900.779730] ucsi_ccg 1-0008: [typec-port0] Cable state:1, cable id:2
[ 1908.760844] ucsi_ccg 1-0008: port0 evt: Source Disabled State Entered

Any assistance would be good.

3

Hi,
Is the issue specific to J512 or J513 ports? Loos like the device is with type-A port. Could you try other USB C to A adapter?

4

No, Usb connected to 2-3 is the Type-C port adjacent to the power jack. I should of mentioned that the issue was solved with a reboot, but this is the 2nd time this style of crash has been noticed, unknown if its the same device.

[ 1883.024237] tegra-xusb 3610000.xhci: entering ELPG
[ 1883.024257] tegra-xusb 3610000.xhci: 2-3 is not suspended: 0a000302
[ 1883.024262] tegra-xusb 3610000.xhci: entering ELPG failed
Can you please look into what these errors indicate, it appears to be specifying a register in the message?

5

Hi,
From the log, it looks like the device is enumerated as SuperSpeed and then disconnected continuously. Seems to be an issue in signal quality. Do you have other C to A adapter for a try?

6

I said in the original post that the above log is captured from me moving the USBC cameras between the two USB C ports continiously to generate the error. Are you confusing that for “disconnected continuously”, or is their something in the log that indicates a disconnect outside of that?

I’m confused on why a C to A adapter is relevant? There was none used in the original issue.

7

Hi,
We are not able to understand the usercase correctly from the log. If the issue can be reproduced on Xavier devkit, please share the steps and we can try to reproduce it with USB3 camera(we have E-Con See3CAM CU135).

8

Okay. So to clarify. I have two USB C cameras plugged in, the one plugged into 2-3 was noticed to not be working, it was not visible using lsusb and then when we went into dmesg we could see it wasnt identified as normal. We then alternated the two cameras back and foward between usb 2-1 and 2-3, which you can see in the log from the serial number of the camera changing between successive plug-ins. Each time though the camera that was plugged into 2-3, again was unable to be detected.

In normal operation this is the only thing we see in dmesg related to the USBC cameras:
[ 13.500568] tegra-xusb 3610000.xhci: Upgrade port 0 to USB3.0
[ 13.500577] tegra-xusb 3610000.xhci: Upgrade port 1 to USB3.0
[ 14.477731] usb 2-1: new SuperSpeed USB device number 2 using tegra-xusb
[ 14.499521] usb usb1: usb_suspend_both: status 0
[ 14.499664] usb 2-1: New USB device found, idVendor=1e10, idProduct=4000
[ 14.499673] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 14.499681] usb 2-1: Product: Blackfly S BFS-U3-50S5C
[ 14.499689] usb 2-1: Manufacturer: FLIR
[ 14.499695] usb 2-1: SerialNumber: 01277BB6
[ 14.793688] usb 2-3: new SuperSpeed USB device number 3 using tegra-xusb
[ 14.815480] usb 2-3: New USB device found, idVendor=1e10, idProduct=4000
[ 14.815495] usb 2-3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 14.815504] usb 2-3: Product: Blackfly S BFS-U3-50S5C
[ 14.815511] usb 2-3: Manufacturer: FLIR
[ 14.815517] usb 2-3: SerialNumber: 0117021A

We can’t reliably replicate the issue, it appears to be a low level issue. Hence my direct questions to the nature of the error lines i highlighted above.

9

These look like exact matches in model. I am curious, do these cameras have any kind of firmware update available from the manufacturer? If the two are exact matches, and if there is not a hardware issue causing signal quality differences, then I would think what is left is firmware (but many such cameras do not have firmware updates), or else something as simple as a cable in less than perfect shape (e.g., stepped on or bent too sharply changing the shape and signal qualities).

10

Hmmm, good thought but I don’t think so. When we’re swapping the cameras from Port to Port, we’re also swapping the cables. And while yes they do both have the same firmware, it would be unusual that the cameras would be the issue if a restart of the Xavier fixes the issue.

11

This is likely correct if the verbose lsusb is an exact match between the two units. Since these are hot plug devices, any difference in their USB information could result in different software triggering firmware upload. I don’t know what the brief “lsusb” shows for those cameras, but an ID will be shown with just a basic “lsusb”, and that ID will have a format similar to “0955:7019”. To get a verbose lsusb (adjust for your ID, this is just an example):
sudo lsusb -d 0955:7019 -vvv

If those two devices show the exact same thing side by side, then the firmware would be loaded similarly on both and there is no need to further consider firmware. If the two differ in some way, then it would be interesting to see what that difference is.

12

Again, since both cameras didn’t work on the same port and both worked on the working port, I would not be investigating the cameras for faults.
Noting that lsusb did show the devices connected to port 2-3 when this was occurring.

If I look at the verbose output now there’s nothing overly interesting with the only difference between them being their serial number.

13

I believe you are correct, but short of a USB analyzer, I don’t know what you’d need to look at next. There may be “/sys” files to examine, but I do not know enough about it to say if this would help or not.

14

Aye, same here in terms of not knowing enough, I was hoping to just get some more information about the error/state messages from the dmesg log I posted and work out from there. But as you can see above it seems that the nvidia moderator also doesn’t know and/or is avoiding the direct question :(

15

Hi,
We have tried to connect USB3 cameras E-Con CU135 or Intel RealSense D435 to J512 or J513 port. Don’t hit the issue. Both cameras can be enumerated as Super Speed successfully.
It would be great if you can share bus trace with USB analyzer.

16

@DaneLLL

  • Please give me a break down of the dmesg logs as I’ve posted above, so far it’s the only information avaliable and i need to know what these messages mean, specifically the ones involving tegra-xhci due to its relation with the on-board usb controller.

  • As stated above, this bug is not reproducable, if it happens again i can take the above requests and attempt stuff like a usb bus trace, but as it stands that information is not useful. Unless you can explain why you want it.

  • Though i have a feeling since there was no device in the lsusb log and since usbmod creates its debug logs based on the same information as lsusb, that there would’ve been nothing to enable a trace, i.e. no /sys/kernel/debug/usb/usbmon/2X file to run a tcptrace on.

  • Plugging in a random camera and saying it works comes across as almost insultive, please don’t do that. I’m aware your cameras work, mine do too for the most part, it’s just this particular bug we’re discussing.