PCIe USB3.1 expansion card doesn't work

Autonomous Machines Jetson & Embedded Systems Jetson TX2
#1

I’m trying to install the following PCIe USB3.1 card on our Jetson TX2 developer board:
https://www.startech.com/Cards-Adapters/USB-3.0/Cards/2-port-usb-3-1-card~PEXUSB312A2#dnlds

The card seems to be installed correctly (it is listed on lspci), but when I plug a USB device it wouldn’t show up in the system. I’ve tried two Logitech C920 webcams and a USB keyboard.

Here is the output of lspci, lsusb and dmesg | grep pcie:
(With a C920 connected to the PCIe card - that doesn’t appear in the system -, a bluetooth mouse dongle and a keyboard plugged to a usb hub connected to the USB board on the development board)

ubuntu@tegra-ubuntu:~$ lspci
00:01.0 PCI bridge: NVIDIA Corporation Device 10e5 (rev a1)
01:00.0 USB controller: ASMedia Technology Inc. ASM1142 USB 3.1 Host Controller

ubuntu@tegra-ubuntu:~$ lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 004: ID 045e:0745 Microsoft Corp. Nano Transceiver v1.0 for Bluetooth
Bus 001 Device 006: ID 045e:0750 Microsoft Corp. Wired Keyboard 600
Bus 001 Device 002: ID 0409:0059 NEC Corp. HighSpeed Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

ubuntu@tegra-ubuntu:~$ dmesg | grep pcie
[    0.349381] node /plugin-manager/fragment-500-pcie-config match with board >=3310-1000-500
[    0.367236] node /plugin-manager/fragment-500-e3325-pcie match with board >=3310-1000-500
[    1.174519] iommu: Adding device 10003000.pcie-controller to group 48
[   13.721153] tegra-pcie 10003000.pcie-controller: 4x1, 1x1 configuration
[   13.731059] tegra-pcie 10003000.pcie-controller: PCIE: Enable power rails
[   13.743621] tegra-pcie 10003000.pcie-controller: probing port 0, using 4 lanes
[   13.756143] tegra-pcie 10003000.pcie-controller: probing port 2, using 1 lanes
[   14.204535] tegra-pcie 10003000.pcie-controller: link 2 down, retrying
[   14.608474] tegra-pcie 10003000.pcie-controller: link 2 down, retrying
[   15.011452] tegra-pcie 10003000.pcie-controller: link 2 down, retrying
[   15.013467] tegra-pcie 10003000.pcie-controller: link 2 down, ignoring
[   15.013760] tegra-pcie 10003000.pcie-controller: PCI host bridge to bus 0000:00
[   15.020645] pcieport 0000:00:01.0: enabling device (0000 -> 0002)
[   15.020711] pcieport 0000:00:01.0: Signaling PME through PCIe PME interrupt
[   15.020717] pcie_pme 0000:00:01.0:pcie01: service driver pcie_pme loaded
[   15.020783] aer 0000:00:01.0:pcie02: service driver aer loaded
[   15.081099] tegra-pcie 10003000.pcie-controller: speed change : Gen-1 -> Gen-2

Any ideas? Do I have to set up the pcie card in any way?

Thanks

Kai

#2

Seems PCIe is working, but what is the verbose output of:

sudo lspci -s 01:00.0 -vvv

I don’t know which of the HUBs listed in lsusb belong to your PCIe version, but notice lsusb has an ID of the format “0409:0059” (example from the NEC HUB…probably not the right HUB). You can get a verbose output of what USB drivers see via (use the actual ID):

sudo lsusb -d 0409:0059 -vvv
#3

The device with the ID 0409:0059 is the hub that is connected to the developer boards USB.

When I disconnect that hub i get:

ubuntu@tegra-ubuntu:~$ lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

I booted the system without the PCIe card:

ubuntu@tegra-ubuntu:~$ lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Not much help. I will attach the verbose output of lspci below.

I’ve found this post:
https://devtalk.nvidia.com/default/topic/985392/usb-devices-not-detected-at-pcie-card-using-ubuntu-16-04/
Maybe I’m out of luck.

ubuntu@tegra-ubuntu:~$ sudo lspci -s 01:00.0 -vvv
01:00.0 USB controller: ASMedia Technology Inc. ASM1142 USB 3.1 Host Controller (prog-if 30 [XHCI])
        Subsystem: ASMedia Technology Inc. ASM1142 USB 3.1 Host Controller
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 128 bytes
        Interrupt: pin A routed to IRQ 379
        Region 0: Memory at 50100000 (64-bit, non-prefetchable) 
        Capabilities: [50] MSI: Enable- Count=1/8 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [68] MSI-X: Enable+ Count=8 Masked-
                Vector table: BAR=0 offset=00002000
                PBA: BAR=0 offset=00002080
        Capabilities: [78] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=55mA PME(D0-,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [80] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <64ns, L1 <2us
                        ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
                LnkCap: Port #1, Speed 5GT/s, Width x2, ASPM L0s L1, Exit Latency L0s <2us, L1 unlimited
                        ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x2, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR+, OBFF Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [100 v1] Virtual Channel
                Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
                Arb:    Fixed- WRR32- WRR64- WRR128-
                Ctrl:   ArbSelect=Fixed
                Status: InProgress-
                VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                        Arb:    Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
                        Ctrl:   Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
                        Status: NegoPending- InProgress-
        Capabilities: [200 v1] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
        Capabilities: [280 v1] #19
        Capabilities: [300 v1] Latency Tolerance Reporting
                Max snoop latency: 0ns
                Max no snoop latency: 0ns
        Kernel driver in use: xhci_hcd
#4

So the good news is that it is capable of PCIe v3 (faster than the slot capability of v2), and is running at the maximum the slot can handle (v2). The PCIe side is running with no error.

The other URL you mention may be related, I don’t know. There is an issue of IOMMU (SMMU is the ARM-specific name) being enabled possibly causing DMA issues (there are some patches for various DMA issues on PCIe). I don’t see errors in the given data, but there are plenty of other places it might cause problems without actually showing up in lspci.

Did you get the “sudo lsusb -vvv” listing for the HUB? This might offer more clues.

Meanwhile, you could try that patch on DMA from here, although it may be useful to see lsusb verbose output first:
https://devtalk.nvidia.com/default/topic/967515/jetson-tx1/l4t-r24-2-pcie-iommu-not-working-when-switches-bridges-are-present/post/4982441/#4982441

Basically your issue is different than PCIe which can’t be detected…your PCIe is working flawlessly, although DMA might be an issue not showing up under PCIe reporting (not detecting a PCIe device tends to imply the control channel is not getting a signal…your system is getting high enough quality signal that it runs at the max speed of the system without throttling back…DMA is something set up separately from PCIe control signals and data lanes). Imagine PCIe works, and USB almost works…the DMA may not be providing any data, but it would still be useful to know more about what USB drivers are thinking (thus “-vvv” for lsusb).

If you do not want to try the DMA patch you could wait for the next release. There hasn’t been any official announcement of that date, but it was stated on forums that there is a good chance of it in July (and it is now July). If not July, then it would still be “soon”. Even so it isn’t guaranteed that DMA is the issue (though it would probably be high on the list if DMA is used).

#5

I think you are right that the other post isn’t really related. I just tried to install the card on a TX1 system, same symptoms. The OP mentioned that everything worked on his TX1.

I’ve never installed a patch, but I think now is as good a time as any to give it a try. It is a fresh system anyway, if anything goes wrong I can flash the system.

Here are the results of sudo lsusb -vvv:

ubuntu@tegra-ubuntu:~$ sudo lsusb -vvv

Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               3.00
  bDeviceClass            9 Hub
  bDeviceSubClass         0 Unused
  bDeviceProtocol         3
  bMaxPacketSize0         9
  idVendor           0x1d6b Linux Foundation
  idProduct          0x0003 3.0 root hub
  bcdDevice            4.04
  iManufacturer           3 Linux 4.4.15-tegra xhci-hcd
  iProduct                2 xHCI Host Controller
  iSerial                 1 3530000.xhci
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           31
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0
    bmAttributes         0xe0
      Self Powered
      Remote Wakeup
    MaxPower                0mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         9 Hub
      bInterfaceSubClass      0 Unused
      bInterfaceProtocol      0 Full speed (or root) hub
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0004  1x 4 bytes
        bInterval              12
        bMaxBurst               0
Hub Descriptor:
  bLength              12
  bDescriptorType      42
  nNbrPorts             3
  wHubCharacteristic 0x000a
    No power switching (usb 1.0)
    Per-port overcurrent protection
  bPwrOn2PwrGood       10 * 2 milli seconds
  bHubContrCurrent      0 milli Ampere
  bHubDecLat          0.0 micro seconds
  wHubDelay             0 nano seconds
  DeviceRemovable    0x00
 Hub Port Status:
   Port 1: 0000.02a0 5Gbps power Rx.Detect
   Port 2: 0000.02a0 5Gbps power Rx.Detect
   Port 3: 0000.02a0 5Gbps power Rx.Detect
Binary Object Store Descriptor:
  bLength                 5
  bDescriptorType        15
  wTotalLength           15
  bNumDeviceCaps          1
  SuperSpeed USB Device Capability:
    bLength                10
    bDescriptorType        16
    bDevCapabilityType      3
    bmAttributes         0x00
    wSpeedsSupported   0x0008
      Device can operate at SuperSpeed (5Gbps)
    bFunctionalitySupport   3
      Lowest fully-functional device speed is SuperSpeed (5Gbps)
    bU1DevExitLat           2 micro seconds
    bU2DevExitLat          64 micro seconds
Device Status:     0x0003
  Self Powered
  Remote Wakeup Enabled

Bus 001 Device 004: ID 045e:0745 Microsoft Corp. Nano Transceiver v1.0 for Bluetooth
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            0 (Defined at Interface level)
  bDeviceSubClass         0
  bDeviceProtocol         0
  bMaxPacketSize0        64
  idVendor           0x045e Microsoft Corp.
  idProduct          0x0745 Nano Transceiver v1.0 for Bluetooth
  bcdDevice            6.34
  iManufacturer           1 Microsoft
  iProduct                2 Microsoft® Nano Transceiver v2.0
  iSerial                 0
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           84
    bNumInterfaces          3
    bConfigurationValue     1
    iConfiguration          0
    bmAttributes         0xa0
      (Bus Powered)
      Remote Wakeup
    MaxPower              100mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         3 Human Interface Device
      bInterfaceSubClass      1 Boot Interface Subclass
      bInterfaceProtocol      1 Keyboard
      iInterface              0
        HID Device Descriptor:
          bLength                 9
          bDescriptorType        33
          bcdHID               1.11
          bCountryCode            0 Not supported
          bNumDescriptors         1
          bDescriptorType        34 Report
          wDescriptorLength      57
         Report Descriptors:
           ** UNAVAILABLE **
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0008  1x 8 bytes
        bInterval               4
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        1
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         3 Human Interface Device
      bInterfaceSubClass      1 Boot Interface Subclass
      bInterfaceProtocol      2 Mouse
      iInterface              0
        HID Device Descriptor:
          bLength                 9
          bDescriptorType        33
          bcdHID               1.11
          bCountryCode            0 Not supported
          bNumDescriptors         1
          bDescriptorType        34 Report
          wDescriptorLength     295
         Report Descriptors:
           ** UNAVAILABLE **
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x82  EP 2 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x000a  1x 10 bytes
        bInterval               1
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        2
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         3 Human Interface Device
      bInterfaceSubClass      0 No Subclass
      bInterfaceProtocol      0 None
      iInterface              0
        HID Device Descriptor:
          bLength                 9
          bDescriptorType        33
          bcdHID               1.11
          bCountryCode            0 Not supported
          bNumDescriptors         1
          bDescriptorType        34 Report
          wDescriptorLength     319
         Report Descriptors:
           ** UNAVAILABLE **
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x83  EP 3 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0020  1x 32 bytes
        bInterval               1
Device Status:     0x0000
  (Bus Powered)

Bus 001 Device 003: ID 045e:0750 Microsoft Corp. Wired Keyboard 600
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               1.10
  bDeviceClass            0 (Defined at Interface level)
  bDeviceSubClass         0
  bDeviceProtocol         0
  bMaxPacketSize0         8
  idVendor           0x045e Microsoft Corp.
  idProduct          0x0750 Wired Keyboard 600
  bcdDevice            1.10
  iManufacturer           1 Microsoft
  iProduct                2 Wired Keyboard 600
  iSerial                 0
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           59
    bNumInterfaces          2
    bConfigurationValue     1
    iConfiguration          0
    bmAttributes         0xa0
      (Bus Powered)
      Remote Wakeup
    MaxPower              100mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         3 Human Interface Device
      bInterfaceSubClass      1 Boot Interface Subclass
      bInterfaceProtocol      1 Keyboard
      iInterface              0
        HID Device Descriptor:
          bLength                 9
          bDescriptorType        33
          bcdHID               1.11
          bCountryCode            0 Not supported
          bNumDescriptors         1
          bDescriptorType        34 Report
          wDescriptorLength      65
         Report Descriptors:
           ** UNAVAILABLE **
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0008  1x 8 bytes
        bInterval              10
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        1
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         3 Human Interface Device
      bInterfaceSubClass      0 No Subclass
      bInterfaceProtocol      0 None
      iInterface              0
        HID Device Descriptor:
          bLength                 9
          bDescriptorType        33
          bcdHID               1.11
          bCountryCode            0 Not supported
          bNumDescriptors         1
          bDescriptorType        34 Report
          wDescriptorLength     106
         Report Descriptors:
           ** UNAVAILABLE **
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x82  EP 2 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0008  1x 8 bytes
        bInterval              10
Device Status:     0x0000
  (Bus Powered)

Bus 001 Device 002: ID 0409:0059 NEC Corp. HighSpeed Hub
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            9 Hub
  bDeviceSubClass         0 Unused
  bDeviceProtocol         1 Single TT
  bMaxPacketSize0        64
  idVendor           0x0409 NEC Corp.
  idProduct          0x0059 HighSpeed Hub
  bcdDevice            1.00
  iManufacturer           0
  iProduct                0
  iSerial                 0
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           25
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0
    bmAttributes         0xe0
      Self Powered
      Remote Wakeup
    MaxPower              100mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         9 Hub
      bInterfaceSubClass      0 Unused
      bInterfaceProtocol      0 Full speed (or root) hub
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0001  1x 1 bytes
        bInterval              12
Hub Descriptor:
  bLength               9
  bDescriptorType      41
  nNbrPorts             4
  wHubCharacteristic 0x00a9
    Per-port power switching
    Per-port overcurrent protection
    TT think time 16 FS bits
    Port indicators
  bPwrOn2PwrGood       50 * 2 milli seconds
  bHubContrCurrent    100 milli Ampere
  DeviceRemovable    0x00
  PortPwrCtrlMask    0xff
 Hub Port Status:
   Port 1: 0000.0303 lowspeed power enable connect
   Port 2: 0000.0100 power
   Port 3: 0000.0100 power
   Port 4: 0000.0103 power enable connect
Device Qualifier (for other device speed):
  bLength                10
  bDescriptorType         6
  bcdUSB               2.00
  bDeviceClass            9 Hub
  bDeviceSubClass         0 Unused
  bDeviceProtocol         0 Full speed (or root) hub
  bMaxPacketSize0        64
  bNumConfigurations      1
Device Status:     0x0001
  Self Powered

Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            9 Hub
  bDeviceSubClass         0 Unused
  bDeviceProtocol         1 Single TT
  bMaxPacketSize0        64
  idVendor           0x1d6b Linux Foundation
  idProduct          0x0002 2.0 root hub
  bcdDevice            4.04
  iManufacturer           3 Linux 4.4.15-tegra xhci-hcd
  iProduct                2 xHCI Host Controller
  iSerial                 1 3530000.xhci
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           25
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0
    bmAttributes         0xe0
      Self Powered
      Remote Wakeup
    MaxPower                0mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         9 Hub
      bInterfaceSubClass      0 Unused
      bInterfaceProtocol      0 Full speed (or root) hub
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0004  1x 4 bytes
        bInterval              12
Hub Descriptor:
  bLength               9
  bDescriptorType      41
  nNbrPorts             4
  wHubCharacteristic 0x000a
    No power switching (usb 1.0)
    Per-port overcurrent protection
    TT think time 8 FS bits
  bPwrOn2PwrGood       10 * 2 milli seconds
  bHubContrCurrent      0 milli Ampere
  DeviceRemovable    0x00
  PortPwrCtrlMask    0xff
 Hub Port Status:
   Port 1: 0000.0000
   Port 2: 0000.0503 highspeed power enable connect
   Port 3: 0000.0100 power
   Port 4: 0000.0100 power
Device Status:     0x0003
  Self Powered
  Remote Wakeup Enabled

Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               3.00
  bDeviceClass            9 Hub
  bDeviceSubClass         0 Unused
  bDeviceProtocol         3
  bMaxPacketSize0         9
  idVendor           0x1d6b Linux Foundation
  idProduct          0x0003 3.0 root hub
  bcdDevice            4.04
  iManufacturer           3 Linux 4.4.15-tegra xhci-hcd
  iProduct                2 xHCI Host Controller
  iSerial                 1 0000:01:00.0
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           31
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0
    bmAttributes         0xe0
      Self Powered
      Remote Wakeup
    MaxPower                0mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         9 Hub
      bInterfaceSubClass      0 Unused
      bInterfaceProtocol      0 Full speed (or root) hub
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0004  1x 4 bytes
        bInterval              12
        bMaxBurst               0
Hub Descriptor:
  bLength              12
  bDescriptorType      42
  nNbrPorts             2
  wHubCharacteristic 0x000a
    No power switching (usb 1.0)
    Per-port overcurrent protection
  bPwrOn2PwrGood       10 * 2 milli seconds
  bHubContrCurrent      0 milli Ampere
  bHubDecLat          0.0 micro seconds
  wHubDelay             0 nano seconds
  DeviceRemovable    0x00
 Hub Port Status:
   Port 1: 0000.02a0 5Gbps power Rx.Detect
   Port 2: 0000.02a0 5Gbps power Rx.Detect
Binary Object Store Descriptor:
  bLength                 5
  bDescriptorType        15
  wTotalLength           15
  bNumDeviceCaps          1
  SuperSpeed USB Device Capability:
    bLength                10
    bDescriptorType        16
    bDevCapabilityType      3
    bmAttributes         0x02
      Latency Tolerance Messages (LTM) Supported
    wSpeedsSupported   0x0008
      Device can operate at SuperSpeed (5Gbps)
    bFunctionalitySupport   3
      Lowest fully-functional device speed is SuperSpeed (5Gbps)
    bU1DevExitLat           0 micro seconds
    bU2DevExitLat           0 micro seconds
Device Status:     0x0001
  Self Powered

Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            9 Hub
  bDeviceSubClass         0 Unused
  bDeviceProtocol         1 Single TT
  bMaxPacketSize0        64
  idVendor           0x1d6b Linux Foundation
  idProduct          0x0002 2.0 root hub
  bcdDevice            4.04
  iManufacturer           3 Linux 4.4.15-tegra xhci-hcd
  iProduct                2 xHCI Host Controller
  iSerial                 1 0000:01:00.0
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           25
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0
    bmAttributes         0xe0
      Self Powered
      Remote Wakeup
    MaxPower                0mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         9 Hub
      bInterfaceSubClass      0 Unused
      bInterfaceProtocol      0 Full speed (or root) hub
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0004  1x 4 bytes
        bInterval              12
Hub Descriptor:
  bLength               9
  bDescriptorType      41
  nNbrPorts             2
  wHubCharacteristic 0x000a
    No power switching (usb 1.0)
    Per-port overcurrent protection
    TT think time 8 FS bits
  bPwrOn2PwrGood       10 * 2 milli seconds
  bHubContrCurrent      0 milli Ampere
  DeviceRemovable    0x00
  PortPwrCtrlMask    0xff
 Hub Port Status:
   Port 1: 0000.0100 power
   Port 2: 0000.0100 power
Device Status:     0x0001
  Self Powered
ubuntu@tegra-ubuntu:~$
#6

Without the “-d 1234:5678” option (the ID used is an arbitrary example, replace with the actual HUB’s ID) it is hard to know which lsusb entry is the HUB in question. Going back to how lsusb identifies devices, there are basically two methods to narrow lsusb results to a specific device. The “-d” option uses the manufacturer ID and manufacturer device. So if you were to connect three HUBs which are all the same model from the same manufacturer, then three entries would show up. You can also limit by the slot the device is registered to via the “-s 001:001” type of option where the first three digits are “Bus” number and the second three digits are “Device” number. The trouble with the “-s” option is that unplugging and replugging a USB device can change the slot…the “-d” option remains constant.

Example, suppose we look at one of your original lsusb results:

Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

In the above all HUBs have the same manufacturer (this is the “1d6b” of “1d5b:####”). Had you listed via “lsusb -d 1d6b:” all of those devices would show up. Looks like the USB2 listings are produce ID 0002 for this manufacturer, and product ID is 0003 for the USB3 version. If these were your only USB devices, and you were to use “-d :0003”, then only the USB3 version would be listed.

You can combine any combination of “-d” and “-s” to narrow down to the one device, and then add the “-vvv” to make it verbose. By any combination of “-d” and “-s” can you identify which of the “lsusb -vvv” entries is the HUB you are interested in? It won’t do much good if I’m looking at the wrong entry.

One major reason for wanting to see this is to know more about whether the USB ports in question are able to throttle back to older standards/speeds required from items like keyboards. A second reason for wanting the more detailed output is just to verify USB is seeing everything it needs to know the HUB is there.

#7

I did lsusb -vvv one time with the PCIe card plugged and one time without and took the difference. I think that should do the trick.

I don’t know how to interpret the output, but I checked the difference to the board’s USB connector (always PCIe card first):

iSerial 1 0000:01:00.0
vs.
iSerial 1 3530000.xhci

bmAttributes 0x02
vs.
bmAttributes 0x00

bU1DevExitLat           0 micro seconds
bU2DevExitLat           0 micro seconds

Device Status: 0x0001
vs.
bU1DevExitLat 2 micro seconds
bU2DevExitLat 64 micro seconds
Device Status: 0x0003

And the internal USB listed 3 ports, the PCIe listed 2.

Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               3.00
  bDeviceClass            9 Hub
  bDeviceSubClass         0 Unused
  bDeviceProtocol         3
  bMaxPacketSize0         9
  idVendor           0x1d6b Linux Foundation
  idProduct          0x0003 3.0 root hub
  bcdDevice            4.04
  iManufacturer           3 Linux 4.4.15-tegra xhci-hcd
  iProduct                2 xHCI Host Controller
  iSerial                 1 0000:01:00.0
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           31
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0
    bmAttributes         0xe0
      Self Powered
      Remote Wakeup
    MaxPower                0mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         9 Hub
      bInterfaceSubClass      0 Unused
      bInterfaceProtocol      0 Full speed (or root) hub
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0004  1x 4 bytes
        bInterval              12
        bMaxBurst               0
Hub Descriptor:
  bLength              12
  bDescriptorType      42
  nNbrPorts             2
  wHubCharacteristic 0x000a
    No power switching (usb 1.0)
    Per-port overcurrent protection
  bPwrOn2PwrGood       10 * 2 milli seconds
  bHubContrCurrent      0 milli Ampere
  bHubDecLat          0.0 micro seconds
  wHubDelay             0 nano seconds
  DeviceRemovable    0x00
 Hub Port Status:
   Port 1: 0000.02a0 5Gbps power Rx.Detect
   Port 2: 0000.02a0 5Gbps power Rx.Detect
Binary Object Store Descriptor:
  bLength                 5
  bDescriptorType        15
  wTotalLength           15
  bNumDeviceCaps          1
  SuperSpeed USB Device Capability:
    bLength                10
    bDescriptorType        16
    bDevCapabilityType      3
    bmAttributes         0x02
      Latency Tolerance Messages (LTM) Supported
    wSpeedsSupported   0x0008
      Device can operate at SuperSpeed (5Gbps)
    bFunctionalitySupport   3
      Lowest fully-functional device speed is SuperSpeed (5Gbps)
    bU1DevExitLat           0 micro seconds
    bU2DevExitLat           0 micro seconds
Device Status:     0x0001
  Self Powered

Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            9 Hub
  bDeviceSubClass         0 Unused
  bDeviceProtocol         1 Single TT
  bMaxPacketSize0        64
  idVendor           0x1d6b Linux Foundation
  idProduct          0x0002 2.0 root hub
  bcdDevice            4.04
  iManufacturer           3 Linux 4.4.15-tegra xhci-hcd
  iProduct                2 xHCI Host Controller
  iSerial                 1 0000:01:00.0
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           25
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0
    bmAttributes         0xe0
      Self Powered
      Remote Wakeup
    MaxPower                0mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         9 Hub
      bInterfaceSubClass      0 Unused
      bInterfaceProtocol      0 Full speed (or root) hub
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0004  1x 4 bytes
        bInterval              12
Hub Descriptor:
  bLength               9
  bDescriptorType      41
  nNbrPorts             2
  wHubCharacteristic 0x000a
    No power switching (usb 1.0)
    Per-port overcurrent protection
    TT think time 8 FS bits
  bPwrOn2PwrGood       10 * 2 milli seconds
  bHubContrCurrent      0 milli Ampere
  DeviceRemovable    0x00
  PortPwrCtrlMask    0xff
 Hub Port Status:
   Port 1: 0000.0100 power
   Port 2: 0000.0100 power
Device Status:     0x0001
  Self Powered
#8

I forgot to mention: in the previous posting I just listed the difference, so that’s the output for the PCIe card.

#9

PCIe will list the number of USB controllers in the card via lspci…each controller can have one or more ports with each port getting a listing in lsusb. I some cases a single port will choose a different controller depending on speed requested at the port versus speeds provided by the controller (USB1.1 and USB2 tend to use the same controller, while USB3 tends to be segregated to its own controller).

I don’t see anything which would stand out as causing a failure for either the USB or the PCIe basic setup and driver function. Unless the SMMU/IOMMU issue is involved it appears this USB card should be working. About the only possibility for why this would fail which is a current known issue would be the SMMU. This does not mean that it couldn’t be something else, but it does seem to be the place to start looking.

Since this driver is not custom, can anyone suggest a way to test if this PCIe card is attempting to use SMMU I/O subject to the possibility of failures which other PCIe cards have seen related to enabling SMMU? One obvious test is to add the PCIe SMMU patch listed in a different thread, but perhaps something like devmem2 or something in dmesg could be used to definitively check if SMMU is involved for this particular card (in which case an apparent lack of USB detection might be a case of SMMU simply not putting data through as expected).

#10

Sorry for the delay, I had/have urgent business with another project. I’m going to apply the patch when I’ve got more time and post my findings.

In the meantime, I want to thank you for your support. It helped me a lot.

#11

Today I’ve found the time to install the patch on the TX1. It didn’t resolve my issue, but something happened, with lsusb I find one more device:

ubuntu@tegra-ubuntu:~$ lsusb
Bus 004 Device 002: ID 0955:09ff NVidia Corp.
...

I think that’s the same device that you see on the host computer when the Jetson is in recovery mode.

I’m convinced now that there is no easy solution. We would be using the card to plug additional USB cams, and this thread implies that there is not much hope:
https://devtalk.nvidia.com/default/topic/1014971/jetson-tx2/verified-pcie-to-usb3/

On the other hand, we have a triple CIS camera set now, and they are great.

Thanks again!

#12

When a TX1 is in recovery mode the ID is 0955:7721 (a TX2 is 0955:7c18). The 0955 part is manufacturer NVIDIA, but device 09ff differs. I’m not where I can verify, but I think what you are seeing is a USB controller which is used internally for its D+/D- pair and only shows up when there is a device connected. I’ll see if I can verify if it is what I think it is tomorrow.

#13

The 0955:09ff of lsusb is a Realtek USB ethernet device. I don’t see this on my TX2 (running the most recent L4T R27.1):

# head -n 1 /etc/nv_tegra_release
# R27 (release), REVISION: 1.0, GCID: 8566447, BOARD: t186ref, EABI: aarch64, DATE: Thu Mar  2 05:14:54 UTC 2017

This does not show on my TX2, though internally I believe the ethernet does go through a USB controller (and it is Realtek)…so it could be the module’s and I also do not have that patch in place. Do you have any USB network device attached? It’s probably a good sign if something shows up which didn’t before, it is progress.

Can anyone from NVIDIA verify if there is a way to know if SMMU is interfering with recognition of his USB3.1 adapter on PCIe? The gist is that PCIe and USB are correctly listing the device, but items plugged into it do not show up…perhaps DMA being incorrect might have an effect on not seeing devices plugged into the USB ports, yet USB and PCIe might not show an error. An alternate means of determining if DMA is involved would be useful since it isn’t his driver.

EDIT: For @kbergmann to verify…

  1. Is there power being delivered to the extra power connector on the USB3.1 card?
  2. What is the source of that power connector supply?

NOTE: This card is compatible with USB2 and newer. Human interface devices (such as a mouse and keyboard) probably won’t show up since they tend to be USB1 or USB1.1. The Logitech C920 webcam would be USB2 and is known to work, so it should show up.

One other note: I see this card says it supports “UASP (USB Attached SCSI Protocol) Rev. 1.0”. Additional drivers may need to be installed to use this, but it should only have an effect on USB mass storage devices.

#14

So initially a SSD that uses UASP will not work? Where could i look for these drivers? Generally could there be any propblems using a SSD with USB3.1?

Arvids

#15

FYI, both PCIe and USB will have drivers for generic data transfer. These drivers announce what hardware is present, and once that is done, their jobs are also done. At that point it is up to drivers which believe they can handle the device to take ownership of the device. The “additional drivers” is a reference to the fact that neither USB nor PCIe provide end device drivers (there is a twist to the USB side of this, but this remains technically correct).

In the previous posts the basis of comments was that PCIe was working, but that does not guarantee end success. I do not know about specific UASP attached SCSI protocols…just that they would need their own driver and getting to a certain point implies PCIe or USB is working. For any particular device you must still research what other drivers are needed. You can use “lsusb” to see if your device is visible to USB…if it is, then other drivers will be needed in the chain of support. If not, then USB itself is failing.

Note that USB3 is backwards compatible to USB3.1 (there are some confusing changes regarding merging of USB3 into USB3.1 where USB3.1 can run at faster speeds but is the same protocol). USB3.1 does not become supported in Linux until kernel version 4.6. The version used is 4.4. USB3 is supported, but not 3.1 (it may be a 3.0 driver works with a 3.1 device, but limits to 5Gbit/s…don’t know).

Most any SSD supporting USB3.1 will work on USB3.0. Unless you have multiple SSDs on the port it is unlikely the SSD can actually use the full USB3.0 bandwidth anyway.