Slow USB performance on J513 after boot

I am using two Flir Blackfly S cameras connected to the USB Type C connectors of the AGX dev kit. During development, I have also been attaching a USB hub to the third USB in order to connect a keyboard and mouse. Now that I am trying to make the system headless and only access the dev kit via Ethernet, I’m encountering a performance problem.

The camera attached to J513 (usb 2-1) cannot run at its full frame rate after a power cycle. The second camera still performs well at 30-40 FPS but the one attached to J513 is only getting ~6FPS. Querying the device via /sys/bus/usb/devices/2-1/speed and version indicates it is recognized as USB 3.1. The only workarounds I have found are to again attach the (unneeded) hub to the J507 port and reboot or to physically disconnect and reconnect the camera with the problem. Either workaround resolves the issue until the system is power-cycled without something attached to J507.

I have tried different cameras and different cables and the problem consistently stays with the port. Also, based on other forum posts, I’ve tried setting usbcore.autosuspend=-1.

I am attaching two captures from dmesg. One is from a fresh boot with the hub attached (when there is no problem) and one when the hub is not attached (and then slow performance on J513). I’ve had a thought regarding the following messages:
tegra-xusb 3610000.xhci: Upgrade port 0 to USB3.0
tegra-xusb 3610000.xhci: Upgrade port 1 to USB3.0
Since the usb2-1 is loaded just after this, could the upgrade process still be in progress resulting in the device actually being loaded up as a USB 2.0 device even though it will report otherwise?
-Is there a way to adjust the boot process to allow more time between the upgrade process and loading of usb2-1?
-Are there any other suggestions on how to troubleshoot or solve this issue?

Any help is greatly appreciated…thanks in advance!

dmesgOKwithHub.txt (69.9 KB) dmesgSlow.txt (67.3 KB)

Hi,
There are two USB3 devices detected. Does the issue happen in connecting single USB3 device to J513? Is it specific to the device? Would be great if you can share information about the USB3 device.

And please share the release version( $ head -1 /etc/nv_tegra_release )

Hello,
Thanks for the reply! The release version output is:

R32 (release), REVISION: 4.3, GCID: 21589087, BOARD: t186ref, EABI: aarch64, DATE: Fri Jun 26 04:34:27 UTC 2020

The behavior is the same when only one camera is attached and on J513. It does not matter which camera is connected. The performance is slow on the port regardless of which camera or which cable. For comparison, I am attaching a dmesg capture of the startup logging when only one camera is attached. (After boot, I had to attach the usb hub and monitor to perform a single camera test. You can see that activity at the 11.60…/387.24… timestamp gap).

I’m also attaching lsusb output to provide more details about the USB cameras.

Thanks!

dmesgJ513only.txt (72.9 KB) lsusbTwoCams.txt (12.6 KB)

Hi,
Could you share the steps to check USB performance? We have E-Con CU135 USB camera. Would like to replicate it with the device.

The initial symptom was seeing the camera connected to J513 issue timeout errors (which are set to trigger when it falls below 20fps) and the other camera was operating at 30-40fps in our own custom application. Then, I used Flir’s SpinView_Qt application to duplicate the problem. The slow performance is still seen only on that port whether there is only that camera connected, or two cameras are connected.

To clarify, when the keyboard/mouse/monitor is needed for testing, I still boot the Xavier with only the cameras and Ethernet attached. Then, I attach the HDMI and USB hub for mouse and keyboard after the system is booted. The slow performance is still present under those circumstances.

Hi,
We connect E-Con CU135 to J513 and can achieve 3840x2160p15 UYVY capture in running 12_camera_v4l2_cuda sample. It is maximum throughput of the camera, and we are not able to verify further cases such as 4Kp30 or 4Kp60 locally.

There is description in Xavier Technical Reference Manual:

9.2.2.1.2 USB 3.1 Ports
USB 3.1 ports support both Generation 1 SuperSpeed USB and Generation 2 SuperSpeedPlus USB 10 Gbps
transfer rates.
Note that due to throughput limitations in the xHCI controller, USB 3.1 port 0 and port 3 are connected to the
same SuperSpeedPlus hub to share 10 Gbps total bandwidth, while USB 3.1 port 1 and port 2 share another 10
Gbps total bandwidth. Note also that in the case of two SuperSpeed devices connected to the same
SuperSpeedPlus hub, due to scheduling policy reasons each SuperSpeed device may not be able sustain the
full SuperSpeed unit bandwidth if the other is device is active.

J513 is USB 3.1 port 0 and the type-A port is USB 3.1 port 3. The two ports share 10Gbps. So one possibility is the device in type-A port occupies certain bandwidth and affects J513.

In our situation, attaching a device to the Type A port is a solution. The performance is lower when nothing is attached to the Type A port. While attaching an extra device is a workaround, we need to be able to power our Xavier and cameras up remotely without having an additional device present.