Not enough host controller resources for new device state

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier
,
1

I have a Jetson AGX to which I’m connecting a board that has 6 sierra USB modems (EM7565). When I turn the modems on, the first four modems turn on fine. The fifth and sixth modems, however, show an error message in dmesg:

[  741.023961] usb 1-4.2.1: new high-speed USB device number 16 using tegra-xusb
[  741.044924] usb 1-4.2.1: config 1 has an invalid interface number: 8 but max is 3
[  741.045119] usb 1-4.2.1: config 1 has no interface number 1
[  741.045986] usb 1-4.2.1: New USB device found, idVendor=1199, idProduct=9091
[  741.045994] usb 1-4.2.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[  741.046001] usb 1-4.2.1: Product: Sierra Wireless EM7565 Qualcomm® Snapdragon™ X16 LTE-A
[  741.046009] usb 1-4.2.1: Manufacturer: Sierra Wireless, Incorporated
[  741.046015] usb 1-4.2.1: SerialNumber: UF94918630021546
[  741.046936] usb 1-4.2.1: Not enough host controller resources for new device state.
[  741.047165] usb 1-4.2.1: can't set config #1, error -12

The problem is not specific to any particular modem, if I bring up just modems 5 and 6, they work.

Is this a known issue?

Thank you
Shachar

2

Just for added info, here is some more info:

jetson$ uname -a
Linux jetson 4.9.140-tegra #1 SMP PREEMPT Thu Sep 24 16:09:59 PDT 2020 aarch64 aarch64 aarch64 GNU/Linux
jetson$ lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 014: ID 1199:9091 Sierra Wireless, Inc. 
Bus 001 Device 015: ID 1199:9091 Sierra Wireless, Inc. 
Bus 001 Device 005: ID 04d8:00dd Microchip Technology, Inc. 
Bus 001 Device 012: ID 1199:9091 Sierra Wireless, Inc. 
Bus 001 Device 017: ID 1199:9091 Sierra Wireless, Inc. 
Bus 001 Device 013: ID 1199:9091 Sierra Wireless, Inc. 
Bus 001 Device 016: ID 1199:9091 Sierra Wireless, Inc. 
Bus 001 Device 004: ID 0424:2517 Standard Microsystems Corp. Hub
Bus 001 Device 003: ID 0424:4041 Standard Microsystems Corp. 
Bus 001 Device 002: ID 0424:2640 Standard Microsystems Corp. USB 2.0 Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
jetson$ lsusb -t
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=tegra-xusb/4p, 10000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=tegra-xusb/4p, 480M
    |__ Port 4: Dev 2, If 0, Class=Hub, Driver=hub/3p, 480M
        |__ Port 1: Dev 3, If 0, Class=Mass Storage, Driver=usb-storage, 480M
        |__ Port 2: Dev 4, If 0, Class=Hub, Driver=hub/7p, 480M
            |__ Port 6: Dev 15, If 0, Class=Vendor Specific Class, Driver=, 480M
            |__ Port 6: Dev 15, If 3, Class=Vendor Specific Class, Driver=, 480M
            |__ Port 6: Dev 15, If 8, Class=Vendor Specific Class, Driver=qmi_wwan, 480M
            |__ Port 6: Dev 15, If 2, Class=Vendor Specific Class, Driver=, 480M
            |__ Port 4: Dev 12, If 8, Class=Vendor Specific Class, Driver=qmi_wwan, 480M
            |__ Port 4: Dev 12, If 2, Class=Vendor Specific Class, Driver=, 480M
            |__ Port 4: Dev 12, If 0, Class=Vendor Specific Class, Driver=, 480M
            |__ Port 4: Dev 12, If 3, Class=Vendor Specific Class, Driver=, 480M
            |__ Port 2: Dev 13, If 0, Class=Vendor Specific Class, Driver=, 480M
            |__ Port 2: Dev 13, If 3, Class=Vendor Specific Class, Driver=, 480M
            |__ Port 2: Dev 13, If 8, Class=Vendor Specific Class, Driver=qmi_wwan, 480M
            |__ Port 2: Dev 13, If 2, Class=Vendor Specific Class, Driver=, 480M
            |__ Port 7: Dev 14, If 2, Class=Vendor Specific Class, Driver=, 480M
            |__ Port 7: Dev 14, If 0, Class=Vendor Specific Class, Driver=, 480M
            |__ Port 7: Dev 14, If 3, Class=Vendor Specific Class, Driver=, 480M
            |__ Port 7: Dev 14, If 8, Class=Vendor Specific Class, Driver=qmi_wwan, 480M
            |__ Port 5: Dev 5, If 1, Class=CDC Data, Driver=cdc_acm, 12M
            |__ Port 5: Dev 5, If 2, Class=Human Interface Device, Driver=i2c-mcp2221, 12M
            |__ Port 5: Dev 5, If 0, Class=Communications, Driver=cdc_acm, 12M

Successful modem’s dmesg looks like this:

[  672.672261] usb 1-4.2.6: new high-speed USB device number 15 using tegra-xusb
[  672.693064] usb 1-4.2.6: config 1 has an invalid interface number: 8 but max is 3
[  672.693271] usb 1-4.2.6: config 1 has no interface number 1
[  672.694105] usb 1-4.2.6: New USB device found, idVendor=1199, idProduct=9091
[  672.694113] usb 1-4.2.6: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[  672.694119] usb 1-4.2.6: Product: Sierra Wireless EM7565 Qualcomm® Snapdragon™ X16 LTE-A
[  672.694129] usb 1-4.2.6: Manufacturer: Sierra Wireless, Incorporated
[  672.694134] usb 1-4.2.6: SerialNumber: UF94918606031546
[  672.700067] qmi_wwan 1-4.2.6:1.8: cdc-wdm3: USB WDM device
[  672.700805] qmi_wwan 1-4.2.6:1.8 wwan3: register 'qmi_wwan' at usb-3610000.xhci-4.2.6, WWAN/QMI device, b6:46:28:9a:39:34
3

The required bandwidth in USB is being exceeded. HUBs of course share a single USB root_hub’s resources. So you would need to move at least part of your devices to use a different root_hub.

Note that if you have a single USB3 camera on a USB2 port (and 480M is USB2), then a single camera would exceed resources. Possibly 2 USB3 cameras on a single USB3 port could exceed resources. Depends on bandwidth requirements.

4

Thank you for the reply, but I don’t think I understand it. Four unconnected modems don’t consume much bandwidth (or, at all). The failures are just too predictable.

We’ve had USB bandwidth exhaustion problems before (never on the modems, mind you). They happen under load, and they happen sporadically. A case where I can predict when things will fail does not ring like a case of bandwidth exhaustion.

And that’s before I mention that the same device, with the same 6 modems, functions fine on other platforms we’ve tried. Even a raspberry Pi 3 managed to connect and dial the 6 modems, using the same USB hub and same cable.

5

You could be correct, but do you happen to know if any of those modems use isochronous mode? You can look at this with a verbose lsusb (requires sudo and adding “-vvv”), and if any of it is isochronous, then it means there is a pre-allocation of bandwidth the device expects. Any inability to reserve bandwidth would be an issue, although I doubt a modem would use isochronous mode (it is probably bulk or interrupt mode).

Also, are these connected via a powered HUB, or is this powered over USB? If for some reason this was related to power delivery instead of bandwidth, then a powered HUB would fix the problem (or at least show it isn’t the problem).

It might end up that there is another limitation NVIDIA would have to comment on:

…perhaps this configuration just has a limitation at a lower value than other systems (don’t know).

6

I’m bumping this, as this is still waiting for a solution. If the Jetson cannot connect to 6 modems, I need an official statement that that is the case.

1 Like
7

Are you using a Developer Kit?
There might be several ODM Carrier boards that easily support your usb requirements

8

I am using NVidia’s developer kit.

Links would be appreciated.

Thank you,
Shachar

10

Hi,
Please check

On Jetson Xavier devkit, there are two type-C ports. You may use type-C to type-A adapter to connect the USB devices.
Amazon.com

11

Hi Shachar,

The Connect Tech Rogue Carrier brings all three USB3.1 ports from the AGX Xavier out via USB Type C connectors:

If you have any questions just let me know or email support@connecttech.com and we can help you out.

-Rob Callaghan

12

Hi Shachar, I am happy to assist you with carrier board and system solutions for the Jetson AGX. You can contact me at +972-3-760-2454. - Einat Nagar, FAE @ CRG Electronics.