USB 3.0 expansion card on Jetson TX1 Does not work

[pejman.habashi]

Hello,

I am trying to use PCI-E expansion slot to add extra USB 3.0 ports to the TX1. I have used an expansion card with four separate xhci controllers (so they are not sharing a bus). Here is the result of lspci:

$ lspci

00:01.0 PCI bridge: NVIDIA Corporation Device 0fae (rev a1)
01:00.0 PCI bridge: Pericom Semiconductor Device 2608
02:01.0 PCI bridge: Pericom Semiconductor Device 2608
02:02.0 PCI bridge: Pericom Semiconductor Device 2608
02:03.0 PCI bridge: Pericom Semiconductor Device 2608
02:04.0 PCI bridge: Pericom Semiconductor Device 2608
03:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)
04:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)
05:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)
06:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)

I assume the last four are the controllers for USB 3.0 Expansion card. The problem is that I can not see any device connected to this expansion card. I have tried with a mouse, a usb stick and a USB 3.0 camera, but none of them are visible with lsusb:

$ lsusb

Bus 011 Device 006: ID 064f:2af9 WIBU-Systems AG 
Bus 011 Device 005: ID 062a:4102 Creative Labs 
Bus 011 Device 004: ID 03f0:034a Hewlett-Packard Elite Keyboard
Bus 011 Device 003: ID 192f:0416 Avago Technologies, Pte. ADNS-5700 Optical Mouse Controller (3-button)
Bus 011 Device 002: ID 1a40:0101 Terminus Technology Inc. Hub
Bus 011 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 009 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 010 Device 002: ID 0955:09ff NVidia Corp. 
Bus 010 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 008 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 007 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

No matter what is the device that I have connected, simply it is not working.

Apparently the driver should be already inside the kernel. Just for the record, here is the system information.

$ uname -a
Linux tegra-ubuntu 3.10.96-tegra #1 SMP PREEMPT Wed Nov 9 19:42:57 PST 2016 aarch64 aarch64 aarch64 GNU/Linux

I have no idea how to fix this problem. I really appreciate any help/information on this matter.

[linuxdev]

PCI is probably working correctly, but go ahead and give a verbose listing of one of the controllers. The first column of the plain lspci is a slot ID, one of the controllers is slot 03:00.0. So this would give verbose listing:

<b>sudo</b> lspci -s 03:00.0

On the lsusb I don’t know which one is integrated, and which is from your card. You might want to see topology with “lsusb -t”. Once you know one is from this controller card via bus and port number (they change depending on order of enumeration, so you need to check right before the next step) you can give a verbose listing via by device bus and slot (if you use ID you will see a listing for every single controller of that type…you only need one). So if for example bus 8 and port 1 is your target:

<b>sudo</b> lsusb -s 8:1 -vvv

You also want to see if there is anything which occurs while following dmesg logs as you add or remove a device from that USB card:

dmesg --follow

Keep in mind that some USB devices are USB1.0, others USB1.1, more USB2, and then there is USB3. A different controller may be interested in different devices depending on the standard followed. Very likely keyboard and mouse are either USB1.0 (speed “1.5M” under “lsusb -t”) or USB1.1 (speed “12M” under “lsusb -t”). Cameras might be USB2 (speed “480M” under “lsusb -t”) or USB3 (speed “5000M” under “lsusb -t”). In the case of a USB3 device it might or might not be able to fall back to USB2. If possible you want to view dmesg while trying plugging in of and removing of a keyboard/mouse at either USB1.0 or USB1.1, plus a device known to be USB2 (you can plug it in to a different USB3 port and see if speed drops back to USB2 speed for a pure USB2 device), and a device known to be USB3. Post if you see any dmesg output from the insert/removal of the USB connection.

[kyler]

Do you see anything in the kernel log when plugging in the devices?

Is the PCI card your own design? The uPD720202 requires firmware to be downloaded, it can either be done by patching the xhci driver or by connecting an NVM device.

[pejman.habashi]

PCI is probably working correctly, but go ahead and give a verbose listing of one of the controllers. The first column of the plain lspci is a slot ID, one of the controllers is slot 03:00.0. So this would give verbose listing:

<b>sudo</b> lspci -s 03:00.0

On the lsusb I don’t know which one is integrated, and which is from your card. You might want to see topology with “lsusb -t”. Once you know one is from this controller card via bus and port number (they change depending on order of enumeration, so you need to check right before the next step) you can give a verbose listing via by device bus and slot (if you use ID you will see a listing for every single controller of that type…you only need one). So if for example bus 8 and port 1 is your target:

<b>sudo</b> lsusb -s 8:1 -vvv

You also want to see if there is anything which occurs while following dmesg logs as you add or remove a device from that USB card:

dmesg --follow

Keep in mind that some USB devices are USB1.0, others USB1.1, more USB2, and then there is USB3. A different controller may be interested in different devices depending on the standard followed. Very likely keyboard and mouse are either USB1.0 (speed “1.5M” under “lsusb -t”) or USB1.1 (speed “12M” under “lsusb -t”). Cameras might be USB2 (speed “480M” under “lsusb -t”) or USB3 (speed “5000M” under “lsusb -t”). In the case of a USB3 device it might or might not be able to fall back to USB2. If possible you want to view dmesg while trying plugging in of and removing of a keyboard/mouse at either USB1.0 or USB1.1, plus a device known to be USB2 (you can plug it in to a different USB3 port and see if speed drops back to USB2 speed for a pure USB2 device), and a device known to be USB3. Post if you see any dmesg output from the insert/removal of the USB connection.

Thanks for your quick and descriptive response. I have tried the commands you mentioned. Here is the verbose version of one of the connected ports.

$ sudo lspci -s 03:00.0 -v

03:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02) (prog-if 30 [XHCI])
        Subsystem: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller
        Flags: bus master, fast devsel, latency 0, IRQ 130
        Memory at 13000000 (64-bit, non-prefetchable) 
        Capabilities: [50] Power Management version 3
        Capabilities: [70] MSI: Enable- Count=1/8 Maskable- 64bit+
        Capabilities: [90] MSI-X: Enable+ Count=8 Masked-
        Capabilities: [a0] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Device Serial Number 13-00-00-00-92-43-14-08
        Capabilities: [150] Latency Tolerance Reporting
        Kernel driver in use: xhci_hcd

and the lsusb with a USB 3.0 camera connected to expansion card: (Not shown in the list)

$ lsusb -t
ls 
/:  Bus 10.Port 1: Dev 1, Class=root_hub, Driver=tegra-xhci/4p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Vendor Specific Class, Driver=r8152, 5000M
/:  Bus 09.Port 1: Dev 1, Class=root_hub, Driver=tegra-xhci/5p, 480M
/:  Bus 08.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 5000M
/:  Bus 07.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M
/:  Bus 06.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 5000M
/:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M
/:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 5000M
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M

I know the bus 10 is the one the board, the same camera connected to onboard usb 3.0 Port the lsusb will provide:

lsusb -t
/:  Bus 10.Port 1: Dev 1, Class=root_hub, Driver=tegra-xhci/4p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Vendor Specific Class, Driver=r8152, 5000M
    |__ Port 2: Dev 3, If 0, Class=Miscellaneous Device, Driver=, 5000M
    |__ Port 2: Dev 3, If 1, Class=Miscellaneous Device, Driver=, 5000M
/:  Bus 09.Port 1: Dev 1, Class=root_hub, Driver=tegra-xhci/5p, 480M
/:  Bus 08.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 5000M
/:  Bus 07.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M
/:  Bus 06.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 5000M
/:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M
/:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 5000M
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M

Interesting part is in dmesg. I am not getting some errors while connecting or disconnecting USB 3.0 Camera, but I do see some red lines on the dmesg output. I have pasted some parts that seems to be related (I am not sure). The whole log would be too chatty.

clear

... 
[    0.000000] Kernel command line: fbcon=map:0 console=tty0 console=ttyS0,115200n8 androidboot.modem=none androidboot.serialno=P2180A00P00940c003fd androidboot.security=non-secure tegraid=21.1.2.0.0 ddr_die=2048M@2048M ddr_die=2048M@4096M section=256M memtype=0 usb_port_owner_info=2 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 debug_uartport=lsport,0 earlyprintk=uart8250-32bit,0x70006000 maxcpus=4 usbcore.old_scheme_first=1 lp0_vec=0x1000@0xff2bf000 nvdumper_reserved=0xff23f000 core_edp_mv=1125 core_edp_ma=4000 gpt android.kerneltype=normal androidboot.touch_vendor_id=0 androidboot.touch_panel_id=63 androidboot.touch_feature=0 androidboot.bootreason=pmc:software_reset,pmic:0x0 net.ifnames=0 root=/dev/mmcblk0p1 rw rootwait usbcore.usbfs_memory_mb=1000 pciehp.pciehp_force=1 usbcore.autosuspend=-1
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
...

[    0.308716] cpuidle: using governor menu
[    0.313383] PMC: Setting PMIC interrupt active-low
[    0.318057] tegra-pmc device create success
...
 0.771862] SCSI subsystem initialized
[    0.775320] libata version 3.00 loaded.
[    0.775446] usbcore: registered new interface driver usbfs
[    0.780789] usbcore: registered new interface driver hub
[    0.786102] usbcore: registered new device driver usb
[    0.791664] gpio wake33 for gpio=188
[    0.796087] regulator_get() failed for (1-0074,vcc), -19
[    0.799910] pca953x 1-0074: no regulator found for vcc. Assuming vcc is always powered
[    0.808489] gpio wake21 for gpio=89
[    0.811374] gpiochip_add: registered GPIOs 1008 to 1023 on device: tca9539
[    0.818353] regulator_get() failed for (1-0077,vcc), -19
[    0.823409] pca953x 1-0077: no regulator found for vcc. Assuming vcc is always powered
[    0.831959] gpio wake36 for gpio=202
[    0.834912] gpiochip_add: registered GPIOs 992 to 1007 on device: tca9539
[    0.842004] i2c i2c-3: of_i2c: modalias failure on /i2c@7000c700/prod-settings
[    0.849599] max77620 4-003c: CID0: 0xae
[    0.852697] max77620 4-003c: CID1: 0x06
[    0.856492] max77620 4-003c: CID2: 0x1b
[    0.860314] max77620 4-003c: CID3: 0x59
[    0.864116] max77620 4-003c: CID4: 0x3c
[    0.867930] max77620 4-003c: CID5: 0x80
[    0.871757] max77620 4-003c: MAX77620 PMIC OTP Version is 0x3C
...

Can you understand what does it mean? Really appreciate it. I can provide full output if it is required.

[linuxdev]

The verbose lspci did not show as much as possible…you used “-v” once, you need three to max it out (“-vvv”). A full verbose response under sudo can give more information.

The camera is USB3, possibly this is one which can’t function under USB2. Your PCIe USB card shows separate controllers for USB2 mode and USB3 mode. A camera which is purely USB3 won’t show up if the USB2 controller is selected. Perhaps it is a case of part of the driver software for the card being there, and part not. Incidentally, lspci will show up just fine and list the USB controller even if the controller does not have a driver.

The USBcore is standard, the rest is unrelated so far as I know (it is possible a Vcc message is related, but I doubt it).

If you can post the fully verbose (“-vvv”) lspci it’ll be easier to know more about whether PCI is fully functioning…I’m thinking it probably is.

You might want to post specs on the USB3 card, I’m thinking PCIe and USB are working correctly up to the point of needing the driver for the USB2 and USB3 controllers within your card.

Do you have a known purely USB2 device you can test? Mouse/keyboard won’t be USB2, and the camera seems to be purely USB3.

[pejman.habashi]

Here is the more detailed version of lspci:

~$ sudo lspci -s 3:00.0 -vvv
03:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02) (prog-if 30 [XH                                                                                                                                  CI])
        Subsystem: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINT                                                                                                                                  x+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx                                                                                                                                  -
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 130
        Region 0: Memory at 13000000 (64-bit, non-prefetchable) 
        Capabilities: [50] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold-)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [70] MSI: Enable- Count=1/8 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [90] MSI-X: Enable+ Count=8 Masked-
                Vector table: BAR=0 offset=00001000
                PBA: BAR=0 offset=00001080
        Capabilities: [a0] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <4us, L1 unlimited
                        ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
                        ExtSynch- ClockPM+ AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR+, OBFF Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [100 v1] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- A                                                                                                                                  CSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- A                                                                                                                                  CSViol-
                UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- A                                                                                                                                  CSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
        Capabilities: [140 v1] Device Serial Number 13-00-00-00-92-43-14-08
        Capabilities: [150 v1] Latency Tolerance Reporting
                Max snoop latency: 0ns
                Max no snoop latency: 0ns
        Kernel driver in use: xhci_hcd

The camera is definitely a USB 3.0 only camera. I had also tried with a two USB sticks, one regular (I assume it should be usb2) and another USB stick which has USB 3.0 support. None of them seems to work with the extension board. The board has four “Renesas / NEC - µPD720202” chipset. The full specs should be found here: https://www.startech.com/ca/Cards-Adapters/USB-3.0/Cards/PCI-Express-USB-3-Card-4-Dedicated-Channels-4-Port~PEXUSB3S44V. This is the board that I do have.

I had run dmesg today, and I noticed some errors (see lines 44, 70, 96 and 122):

...
[    3.267259] NET: Registered protocol family 24
[    3.267308] usbcore: registered new interface driver r8152
[    3.267337] usbcore: registered new interface driver asix
[    3.267361] usbcore: registered new interface driver ax88179_178a
[    3.267382] usbcore: registered new interface driver cdc_ether
[    3.267417] usbcore: registered new interface driver smsc95xx
[    3.267442] usbcore: registered new interface driver net1080
[    3.267465] usbcore: registered new interface driver cdc_subset
[    3.267486] usbcore: registered new interface driver zaurus
[    3.267526] usbcore: registered new interface driver cdc_ncm
[    3.267541] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    3.267811] ehci-pci: EHCI PCI platform driver
[    3.267946] xhci_hcd 0000:03:00.0: xHCI Host Controller
[    3.267963] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 1
[    4.138510] tegradc tegradc.1: hdmi: plugged
[    4.146391] smmu_dump_pagetable(): fault_address=0x00000000fdd04000 pa=0xffffffffffffffff bytes=ffffffffffffffff #pte=0 in L2
[    4.160816] mc-err: (0) csr_afir: EMEM decode error on PDE or PTE entry
[    4.170476] mc-err:   status = 0x6000000e; addr = 0xfdd04000
[    4.179217] mc-err:   secure: no, access-type: read, SMMU fault: nr-nw-s
[    4.190126] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[    4.200158] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    4.210641] usb usb1: Product: xHCI Host Controller
[    4.218823] usb usb1: Manufacturer: Linux 3.10.96-tegra xhci_hcd
[    4.228170] usb usb1: SerialNumber: 0000:03:00.0
[    4.236415] xHCI xhci_add_endpoint called for root hub
[    4.236423] xHCI xhci_check_bandwidth called for root hub
[    4.236510] hub 1-0:1.0: USB hub found
[    4.243675] hub 1-0:1.0: 1 port detected
[    4.251161] xhci_hcd 0000:03:00.0: xHCI Host Controller
[    4.259864] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 2
[    4.270846] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003
[    4.281196] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    4.291994] usb usb2: Product: xHCI Host Controller
[    4.300450] usb usb2: Manufacturer: Linux 3.10.96-tegra xhci_hcd
[    4.310005] usb usb2: SerialNumber: 0000:03:00.0
[    4.318324] xHCI xhci_add_endpoint called for root hub
[    4.318330] xHCI xhci_check_bandwidth called for root hub
[    4.318417] hub 2-0:1.0: USB hub found
[    4.325661] hub 2-0:1.0: 1 port detected
[    4.407979] xhci_hcd 0000:04:00.0: xHCI Host Controller
[    4.416743] xhci_hcd 0000:04:00.0: new USB bus registered, assigned bus number 3
[    4.434553] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002
[    4.436437] smmu_dump_pagetable(): fault_address=0x00000000fdd0e000 pa=0xffffffffffffffff bytes=ffffffffffffffff #pte=0 in L2
[    4.436443] mc-err: (0) csr_afir: EMEM decode error on PDE or PTE entry
[    4.436448] mc-err:   status = 0x6000000e; addr = 0xfdd0e000
[    4.436453] mc-err:   secure: no, access-type: read, SMMU fault: nr-nw-s
[    4.490192] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    4.501210] usb usb3: Product: xHCI Host Controller
[    4.509868] usb usb3: Manufacturer: Linux 3.10.96-tegra xhci_hcd
[    4.519661] usb usb3: SerialNumber: 0000:04:00.0
[    4.528261] xHCI xhci_add_endpoint called for root hub
[    4.528268] xHCI xhci_check_bandwidth called for root hub
[    4.528349] hub 3-0:1.0: USB hub found
[    4.535846] hub 3-0:1.0: 1 port detected
[    4.543618] xhci_hcd 0000:04:00.0: xHCI Host Controller
[    4.552564] xhci_hcd 0000:04:00.0: new USB bus registered, assigned bus number 4
[    4.563742] usb usb4: New USB device found, idVendor=1d6b, idProduct=0003
[    4.574224] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    4.585103] usb usb4: Product: xHCI Host Controller
[    4.593599] usb usb4: Manufacturer: Linux 3.10.96-tegra xhci_hcd
[    4.603239] usb usb4: SerialNumber: 0000:04:00.0
[    4.611641] xHCI xhci_add_endpoint called for root hub
[    4.611647] xHCI xhci_check_bandwidth called for root hub
[    4.611724] hub 4-0:1.0: USB hub found
[    4.619037] hub 4-0:1.0: 1 port detected
[    4.697974] xhci_hcd 0000:05:00.0: xHCI Host Controller
[    4.706726] xhci_hcd 0000:05:00.0: new USB bus registered, assigned bus number 5
[    4.724377] usb usb5: New USB device found, idVendor=1d6b, idProduct=0002
[    4.726358] smmu_dump_pagetable(): fault_address=0x00000000fdd18000 pa=0xffffffffffffffff bytes=ffffffffffffffff #pte=0 in L2
[    4.726363] mc-err: (0) csr_afir: EMEM decode error on PDE or PTE entry
[    4.726368] mc-err:   status = 0x6000000e; addr = 0xfdd18000
[    4.726372] mc-err:   secure: no, access-type: read, SMMU fault: nr-nw-s
[    4.779492] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    4.790417] usb usb5: Product: xHCI Host Controller
[    4.798988] usb usb5: Manufacturer: Linux 3.10.96-tegra xhci_hcd
[    4.808704] usb usb5: SerialNumber: 0000:05:00.0
[    4.817216] xHCI xhci_add_endpoint called for root hub
[    4.817222] xHCI xhci_check_bandwidth called for root hub
[    4.817302] hub 5-0:1.0: USB hub found
[    4.824785] hub 5-0:1.0: 1 port detected
[    4.832574] xhci_hcd 0000:05:00.0: xHCI Host Controller
[    4.841578] xhci_hcd 0000:05:00.0: new USB bus registered, assigned bus number 6
[    4.852886] usb usb6: New USB device found, idVendor=1d6b, idProduct=0003
[    4.863564] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    4.874704] usb usb6: Product: xHCI Host Controller
[    4.883521] usb usb6: Manufacturer: Linux 3.10.96-tegra xhci_hcd
[    4.893521] usb usb6: SerialNumber: 0000:05:00.0
[    4.902285] xHCI xhci_add_endpoint called for root hub
[    4.902291] xHCI xhci_check_bandwidth called for root hub
[    4.902370] hub 6-0:1.0: USB hub found
[    4.910072] hub 6-0:1.0: 1 port detected
[    4.987982] xhci_hcd 0000:06:00.0: xHCI Host Controller
[    4.997160] xhci_hcd 0000:06:00.0: new USB bus registered, assigned bus number 7
[    5.015262] usb usb7: New USB device found, idVendor=1d6b, idProduct=0002
[    5.017208] smmu_dump_pagetable(): fault_address=0x00000000fdd22000 pa=0xffffffffffffffff bytes=ffffffffffffffff #pte=0 in L2
[    5.017213] mc-err: (0) csr_afir: EMEM decode error on PDE or PTE entry
[    5.017217] mc-err:   status = 0x6000000e; addr = 0xfdd22000
[    5.017221] mc-err:   secure: no, access-type: read, SMMU fault: nr-nw-s
[    5.071873] usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    5.083054] usb usb7: Product: xHCI Host Controller
[    5.091850] usb usb7: Manufacturer: Linux 3.10.96-tegra xhci_hcd
[    5.101789] usb usb7: SerialNumber: 0000:06:00.0
[    5.110535] xHCI xhci_add_endpoint called for root hub
[    5.110542] xHCI xhci_check_bandwidth called for root hub
[    5.110633] hub 7-0:1.0: USB hub found
[    5.118330] hub 7-0:1.0: 1 port detected
[    5.126300] xhci_hcd 0000:06:00.0: xHCI Host Controller
[    5.135463] xhci_hcd 0000:06:00.0: new USB bus registered, assigned bus number 8
[    5.146895] usb usb8: New USB device found, idVendor=1d6b, idProduct=0003
[    5.157645] usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    5.168815] usb usb8: Product: xHCI Host Controller
[    5.177603] usb usb8: Manufacturer: Linux 3.10.96-tegra xhci_hcd
[    5.187541] usb usb8: SerialNumber: 0000:06:00.0
[    5.196281] xHCI xhci_add_endpoint called for root hub
[    5.196287] xHCI xhci_check_bandwidth called for root hub
[    5.196366] hub 8-0:1.0: USB hub found
[    5.204047] hub 8-0:1.0: 1 port detected
[    5.278147] tegra_xusb_read_usb_calib: usb_calib0 = 0x0a20a413
[    5.289404] tegra-xhci tegra-xhci: XUSB device id = 0xfac (T210)
[    5.299979] tegra-xhci tegra-xhci: Firmware File: tegra21x_xusb_firmware (124416 Bytes)
[    5.312167] tegra-xhci tegra-xhci: Firmware DMA Memory: dma 0x0000000080040000 mapped 0xffffff8000359000 (124416 Bytes)
[    5.312447] usbcore: registered new interface driver cdc_acm
[    5.312449] cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
[    5.312485] usbcore: registered new interface driver cdc_wdm
[    5.312607] usbcore: registered new interface driver usb-storage
[    5.312697] usbcore: registered new interface driver usbserial
[    5.312728] usbcore: registered new interface driver ftdi_sio
[    5.312749] usbserial: USB Serial support registered for FTDI USB Serial Device
[    5.312917] usbcore: registered new interface driver option
[    5.312934] usbserial: USB Serial support registered for GSM modem (1-port)
[    5.313143] usbcore: registered new interface driver pl2303
[    5.313160] usbserial: USB Serial support registered for pl2303
[    5.313204] usbcore: registered new interface driver nvshieldled
[    5.313207] Nvidia Tegra High-Speed USB Device Controller driver (Apr 30, 2012)
[    5.313483] tegra USB phy - inst[0] platform info:
[    5.313485] port_otg: yes
[    5.313486] has_hostpc: yes
[    5.313488] phy_interface: USB_PHY_INTF_UTMI
[    5.313489] op_mode: TEGRA_USB_OPMODE_DEVICE
[    5.313491] qc2_voltage: 0
[    5.313492] vbus_pmu_irq: 0
[    5.313494] charging: enabled
[    5.313495] remote_wakeup: disabled
[    5.318282] regulator_get() failed for (tegra-udc.0,usb_bat_chg), -19
[    5.318290] tegra-udc tegra-udc.0: usb_bat_chg regulator not registered: USB charging will not be enabled
[    5.318672] usbcore: registered new interface driver xpad
[    5.318704] usbcore: registered new interface driver usb_acecad
[    5.318734] usbcore: registered new interface driver aiptek
[    5.318763] usbcore: registered new interface driver gtco
[    5.318794] usbcore: registered new interface driver hanwang
[    5.318825] usbcore: registered new interface driver kbtab
[    5.318874] usbcore: registered new interface driver wacom
[    5.400775] max77620-rtc max77620-rtc.1: rtc core: registered max77620-rtc as rtc1
[    5.621637] tegra-xhci tegra-xhci: num_hsic_port 0
[    5.632091] tegra-xhci tegra-xhci: Firmware timestamp: 2016-06-16 13:21:43 UTC, Version: 50.16 release, Falcon state 0x20
[    5.646728] tegra-xhci tegra-xhci: Nvidia xHCI Host Controller
[    5.656181] tegra-xhci tegra-xhci: new USB bus registered, assigned bus number 9
[    5.667917] tegra-xhci tegra-xhci: irq 71, io mem 0x70090000
[    5.677238] usb usb9: New USB device found, idVendor=1d6b, idProduct=0002
[    5.678082] Wake51 for irq=118
[    5.678085] Enabling wake51
[    5.678378] Wake16 for irq=34
[    5.678380] Enabling wake16
[    5.679102] tegra-rtc 7000e000.rtc: rtc core: registered tegra-rtc as rtc0
[    5.679321] i2c /dev entries driver
[    5.680939] [OV5693]: probing v4l2 sensor.
[    5.681223] ov5693 6-0036: Failed to allocate eeprom reg map: -22
[    5.748067] usb usb9: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    5.758789] usb usb9: Product: Nvidia xHCI Host Controller
[    5.767725] usb usb9: Manufacturer: Linux 3.10.96-tegra tegra-xhci
[    5.777365] usb usb9: SerialNumber: tegra-xhci
[    5.785482] xHCI xhci_add_endpoint called for root hub
[    5.785489] xHCI xhci_check_bandwidth called for root hub
[    5.785578] hub 9-0:1.0: USB hub found
[    5.792780] hub 9-0:1.0: 5 ports detected
[    5.847917] tegra-xhci tegra-xhci: Nvidia xHCI Host Controller
[    5.857231] tegra-xhci tegra-xhci: new USB bus registered, assigned bus number 10
[    5.868300] usb usb10: New USB device found, idVendor=1d6b, idProduct=0003
[    5.878713] usb usb10: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    5.889515] usb usb10: Product: Nvidia xHCI Host Controller
[    5.898577] usb usb10: Manufacturer: Linux 3.10.96-tegra tegra-xhci
[    5.908330] usb usb10: SerialNumber: tegra-xhci
[    5.916540] xHCI xhci_add_endpoint called for root hub
[    5.916546] xHCI xhci_check_bandwidth called for root hub
[    5.916623] hub 10-0:1.0: USB hub found
[    5.923948] hub 10-0:1.0: 4 ports detected
[    5.987968] Wake44 for irq=81
[    5.994387] Wake40 for irq=53
[    6.000745] Wake41 for irq=53
[    6.007057] Wake42 for irq=53
[    6.013320] Wake43 for irq=53
[    6.358269] usb 10-1: new SuperSpeed USB device number 2 using tegra-xhci
[    6.389246] usb 10-1: New USB device found, idVendor=0955, idProduct=09ff
[    6.399280] usb 10-1: New USB device strings: Mfr=1, Product=2, SerialNumber=6
[    6.409702] usb 10-1: Product: USB 10/100/1000 LAN
[    6.417701] usb 10-1: Manufacturer: Nvidia
[    6.425046] usb 10-1: SerialNumber: 000001000000
[    6.514675] virtual_init
[    6.520983] [IMX135] sensor driver loading
[    6.528345] [imx132] sensor driver loading
[    6.535609] [ar0261] sensor driver loading
[    6.542874] ov7695 sensor driver loading
[    6.549946] mt9m114 sensor driver loading
[    6.557130] [ov10823] sensor driver loading
[    6.564375] usb 10-1: reset SuperSpeed USB device number 2 using tegra-xhci
[    6.564569] usbcore: registered new interface driver uvcvideo
[    6.564571] USB Video Class driver (1.1.1)
[    6.592117] max77620-power-off max77620-power-off.4: Event recorder REG_NVERC : 0x0
[    6.603358] FAN dev name: pwm-fan
[    6.609906] FAN:gpio request success.
[

...

[ 1363.820248] [09-13 11:59:10.168] dhd_dump_eapol_4way_message: ETHER_TYPE_802_1X [TX] : M4 of 4way
[ 1370.329692] smmu_dump_pagetable(): fault_address=0x0000000000000000 pa=0xffffffffffffffff bytes=ffffffffffffffff #pte=0 in L2
[ 1370.341317] mc-err: (0) csw_afiw: EMEM decode error on PDE or PTE entry
[ 1370.348176] mc-err:   status = 0x60010031; addr = 0x00000000
[ 1370.354023] mc-err:   secure: no, access-type: write, SMMU fault: nr-nw-s
[ 1413.593431] [09-13 11:59:59.941] wl_cfg80211_disconnect: Reason 2
[ 1413.613178]  wl_is_linkdown : Link down Reason : WLC_E_LINK
[

Again I do not know if the errors are related or not, but the last one (line 217) happen when I was connecting the USB 3.0 thumb drive. I hope this shed some light to the problem. I really appreciate your time.

[linuxdev]

It looks like the PCIe side is working without issue, and runs at the full PCIe v2 speed.

The USB side is using driver xhci_hcd, so the controllers in your card are recognized and a driver installed.

For debugging purposes it seems information regarding the plug in or unplug event of a device on the USB connector is not making its way to the xhci_hcd driver, or else the xhci_hcd driver ignores the event as not being relevant. If DMA is used I could see the possibility of DMA misconfiguration causing a cut in data, but perhaps there would be other DMA errors. What do you get from this:

dmesg | egrep -i '(dma[usb)'

For debugging, if you run “dmesg --follow” and plug in/unplug a keyboard or mouse (these are USB1.0 or 1.1, drivers would differ or behave differently), does anything show up?

What is the output from “lsmod”? If possible, can you connect this card to another Linux box and see if it works…and then see what lsmod shows when it works? There may be another driver required on top of xhci_hcd (I have not researched this driver).

Also, can you look at any Linux documentation for this card and see if there is a firmware requirement? If firmware is missing you might get the syndrom “the lights are on, but nobody is home”. Drivers may more or less become inert if firmware is required and the firmware is missing.

Of the above this error message tends to support a DMA issue (and it wouldn’t be surprising to see a high performance USB3 card using DMA):

[    4.434553] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002
[    4.436437] smmu_dump_pagetable(): fault_address=0x00000000fdd0e000 pa=0xffffffffffffffff bytes=ffffffffffffffff #pte=0 in L2
[    4.436443] mc-err: (0) csr_afir: EMEM decode error on PDE or PTE entry
[    4.436448] mc-err:   status = 0x6000000e; addr = 0xfdd0e000
[    4.436453] mc-err:   secure: no, access-type: read, SMMU fault: nr-nw-s

So if firmware is not an issue you are probably seeing DMA being used with the wrong address. Can someone from NVIDIA suggest a way to be certain if the PCIe USB card driver is using DMA incorrectly with the wrong address translation?

[pejman.habashi]

I got nothing out of

dmesg | egrep -i '(dma[usb])'

I have tried

dmesg | egrep -i 'dma'

I got:

[    0.284882] DMA: preallocated 1024 KiB pool for atomic coherent allocations
[    1.306905] tegra-carveouts tegra-carveouts.23: iram :dma coherent mem declare 0x0000000040001000,258048
[    3.076331] tegra-apbdma tegra-apbdma: Tegra20 APB DMA driver register 32 channels
[    3.083217] tegra-adma tegra210-adma: Tegra210 ADMA driver register 10 channels
[    3.152933]  (null): coherent DMA mask is unset
[    5.312167] tegra-xhci tegra-xhci: Firmware DMA Memory: dma 0x0000000080040000 mapped 0xffffff8000359000 (124416 Bytes)
[    7.237917] mmc0: SDHCI controller on sdhci-tegra.3 [sdhci-tegra.3] using ADMA
[    7.377917] mmc1: SDHCI controller on sdhci-tegra.1 [sdhci-tegra.1] using ADMA
[    7.527923] mmc2: SDHCI controller on sdhci-tegra.0 [sdhci-tegra.0] using ADMA
[    8.824721] tegra210-ape-admaif tegra210-admaif: ASoC: Failed to create codec debugfs directory
[    8.881596] tegra-snd-t210ref-mobile-rt565x sound.27:  ADMAIF1 <-> ADMAIF1 mapping ok
[    8.893783] tegra-snd-t210ref-mobile-rt565x sound.27:  ADMAIF2 <-> ADMAIF2 mapping ok
[    8.905768] tegra-snd-t210ref-mobile-rt565x sound.27:  ADMAIF3 <-> ADMAIF3 mapping ok
[    8.917621] tegra-snd-t210ref-mobile-rt565x sound.27:  ADMAIF4 <-> ADMAIF4 mapping ok
[    8.929438] tegra-snd-t210ref-mobile-rt565x sound.27:  ADMAIF5 <-> ADMAIF5 mapping ok
[    8.941193] tegra-snd-t210ref-mobile-rt565x sound.27:  ADMAIF6 <-> ADMAIF6 mapping ok
[    8.952964] tegra-snd-t210ref-mobile-rt565x sound.27:  ADMAIF7 <-> ADMAIF7 mapping ok
[    8.964707] tegra-snd-t210ref-mobile-rt565x sound.27:  ADMAIF8 <-> ADMAIF8 mapping ok
[    8.976462] tegra-snd-t210ref-mobile-rt565x sound.27:  ADMAIF9 <-> ADMAIF9 mapping ok
[    8.988223] tegra-snd-t210ref-mobile-rt565x sound.27:  ADMAIF10 <-> ADMAIF10 mapping ok
[12646.178814] sdhci: ADMA Err[0x054]: 0x00000000 | ADMA Ptr[0x058]: 0xfdd28010

Regarding the firmware, I have not installed any firmware in this regard. I don’t know where I should get one if it is required.

I do not have access to a normal Unix machine but the lsmod on jetson TX1 resulted in:

~$ lsmod
Module                  Size  Used by
bnep                   14896  2
bcmdhd               7465032  0
cfg80211              452899  1 bcmdhd
bluedroid_pm           11420  0

dmesg when I connected mouse, I got this error:

mc-err: Too many MC errors; throttling prints

After some googling I encounter: https://devtalk.nvidia.com/default/topic/967515/l4t-r24-2-pcie-iommu-not-working-when-switches-bridges-are-present/ I am not sure but it seems similar to DMA address translation you mentioned. Could it be solved by upgrading something?

[linuxdev]

I got nothing out of
…

dmesg | egrep -i ‘(dma[usb])’

The “usb” is not in brackets, it is just a pipe symbol between (the pipe character is a regular expression “logical OR”, the square brackets in a regular expression limits it to requiring one of those characters):

dmesg | egrep -i '(dma<b>|</b>usb)'

I think sdhci can be ignored for these purposes.

This one is kind of a smoking gun:

mc-err: Too many MC errors; throttling prints

MC is the memory controller, and I suspect that if there is no DMA this won’t occur (I’m not sure if there is a means on command line to temporarily tell the driver to not use DMA…but if temporary DMA disable is possible it would make a good test). Once more, it seems someone from NVIDIA will have to provide a means to definitively determine if the reason for failure is if there is a DMA issue going on…it seems likely.

[pejman.habashi]

dmesg | egrep -i '(dma<b>|</b>usb)'

OOPS! here we go:

$ dmesg | egrep -i '(dma|usb)'
[    0.775721] usbcore: registered new interface driver usbfs
[    0.781066] usbcore: registered new interface driver hub
[    0.786385] usbcore: registered new device driver usb
[    1.162465] usb-vbus1: 5000 mV ; Rail OFF
[    1.166308] usb-vbus1: supplied by vdd-3v3
[    1.170547] usb-vbus3: 5000 mV ; Rail OFF
[    1.174369] usb-vbus3: supplied by vdd-3v3
[    1.215046] en-usb-vbus2: ; Rail OFF
[    1.306905] tegra-carveouts tegra-carveouts.23: iram :dma coherent mem declare 0x0000000040001000,258048
[    1.451064] tegra_xusb_padctl_phy_enable: init pex uphy pll
[    3.076331] tegra-apbdma tegra-apbdma: Tegra20 APB DMA driver register 32 channels
[    3.083217] tegra-adma tegra210-adma: Tegra210 ADMA driver register 10 channels
[    3.152933]  (null): coherent DMA mask is unset
[    3.267308] usbcore: registered new interface driver r8152
[    3.267337] usbcore: registered new interface driver asix
[    3.267361] usbcore: registered new interface driver ax88179_178a
[    3.267382] usbcore: registered new interface driver cdc_ether
[    3.267417] usbcore: registered new interface driver smsc95xx
[    3.267442] usbcore: registered new interface driver net1080
[    3.267465] usbcore: registered new interface driver cdc_subset
[    3.267486] usbcore: registered new interface driver zaurus
[    3.267526] usbcore: registered new interface driver cdc_ncm
[    3.267541] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    3.267963] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 1
[    4.190126] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[    4.200158] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    4.210641] usb usb1: Product: xHCI Host Controller
[    4.218823] usb usb1: Manufacturer: Linux 3.10.96-tegra xhci_hcd
[    4.228170] usb usb1: SerialNumber: 0000:03:00.0
[    4.236510] hub 1-0:1.0: USB hub found
[    4.259864] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 2
[    4.270846] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003
[    4.281196] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    4.291994] usb usb2: Product: xHCI Host Controller
[    4.300450] usb usb2: Manufacturer: Linux 3.10.96-tegra xhci_hcd
[    4.310005] usb usb2: SerialNumber: 0000:03:00.0
[    4.318417] hub 2-0:1.0: USB hub found
[    4.416743] xhci_hcd 0000:04:00.0: new USB bus registered, assigned bus number 3
[    4.434553] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002
[    4.490192] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    4.501210] usb usb3: Product: xHCI Host Controller
[    4.509868] usb usb3: Manufacturer: Linux 3.10.96-tegra xhci_hcd
[    4.519661] usb usb3: SerialNumber: 0000:04:00.0
[    4.528349] hub 3-0:1.0: USB hub found
[    4.552564] xhci_hcd 0000:04:00.0: new USB bus registered, assigned bus number 4
[    4.563742] usb usb4: New USB device found, idVendor=1d6b, idProduct=0003
[    4.574224] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    4.585103] usb usb4: Product: xHCI Host Controller
[    4.593599] usb usb4: Manufacturer: Linux 3.10.96-tegra xhci_hcd
[    4.603239] usb usb4: SerialNumber: 0000:04:00.0
[    4.611724] hub 4-0:1.0: USB hub found
[    4.706726] xhci_hcd 0000:05:00.0: new USB bus registered, assigned bus number 5
[    4.724377] usb usb5: New USB device found, idVendor=1d6b, idProduct=0002
[    4.779492] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    4.790417] usb usb5: Product: xHCI Host Controller
[    4.798988] usb usb5: Manufacturer: Linux 3.10.96-tegra xhci_hcd
[    4.808704] usb usb5: SerialNumber: 0000:05:00.0
[    4.817302] hub 5-0:1.0: USB hub found
[    4.841578] xhci_hcd 0000:05:00.0: new USB bus registered, assigned bus number 6
[    4.852886] usb usb6: New USB device found, idVendor=1d6b, idProduct=0003
[    4.863564] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    4.874704] usb usb6: Product: xHCI Host Controller
[    4.883521] usb usb6: Manufacturer: Linux 3.10.96-tegra xhci_hcd
[    4.893521] usb usb6: SerialNumber: 0000:05:00.0
[    4.902370] hub 6-0:1.0: USB hub found
[    4.997160] xhci_hcd 0000:06:00.0: new USB bus registered, assigned bus number 7
[    5.015262] usb usb7: New USB device found, idVendor=1d6b, idProduct=0002
[    5.071873] usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    5.083054] usb usb7: Product: xHCI Host Controller
[    5.091850] usb usb7: Manufacturer: Linux 3.10.96-tegra xhci_hcd
[    5.101789] usb usb7: SerialNumber: 0000:06:00.0
[    5.110633] hub 7-0:1.0: USB hub found
[    5.135463] xhci_hcd 0000:06:00.0: new USB bus registered, assigned bus number 8
[    5.146895] usb usb8: New USB device found, idVendor=1d6b, idProduct=0003
[    5.157645] usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    5.168815] usb usb8: Product: xHCI Host Controller
[    5.177603] usb usb8: Manufacturer: Linux 3.10.96-tegra xhci_hcd
[    5.187541] usb usb8: SerialNumber: 0000:06:00.0
[    5.196366] hub 8-0:1.0: USB hub found
[    5.278147] tegra_xusb_read_usb_calib: usb_calib0 = 0x0a20a413
[    5.289404] tegra-xhci tegra-xhci: XUSB device id = 0xfac (T210)
[    5.299979] tegra-xhci tegra-xhci: Firmware File: tegra21x_xusb_firmware (124416 Bytes)
[    5.312167] tegra-xhci tegra-xhci: Firmware DMA Memory: dma 0x0000000080040000 mapped 0xffffff8000359000 (124416 Bytes)
[    5.312447] usbcore: registered new interface driver cdc_acm
[    5.312449] cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
[    5.312485] usbcore: registered new interface driver cdc_wdm
[    5.312607] usbcore: registered new interface driver usb-storage
[    5.312697] usbcore: registered new interface driver usbserial
[    5.312728] usbcore: registered new interface driver ftdi_sio
[    5.312749] usbserial: USB Serial support registered for FTDI USB Serial Device
[    5.312917] usbcore: registered new interface driver option
[    5.312934] usbserial: USB Serial support registered for GSM modem (1-port)
[    5.313143] usbcore: registered new interface driver pl2303
[    5.313160] usbserial: USB Serial support registered for pl2303
[    5.313204] usbcore: registered new interface driver nvshieldled
[    5.313207] Nvidia Tegra High-Speed USB Device Controller driver (Apr 30, 2012)
[    5.313483] tegra USB phy - inst[0] platform info:
[    5.313488] phy_interface: USB_PHY_INTF_UTMI
[    5.313489] op_mode: TEGRA_USB_OPMODE_DEVICE
[    5.318282] regulator_get() failed for (tegra-udc.0,usb_bat_chg), -19
[    5.318290] tegra-udc tegra-udc.0: usb_bat_chg regulator not registered: USB charging will not be enabled
[    5.318672] usbcore: registered new interface driver xpad
[    5.318704] usbcore: registered new interface driver usb_acecad
[    5.318734] usbcore: registered new interface driver aiptek
[    5.318763] usbcore: registered new interface driver gtco
[    5.318794] usbcore: registered new interface driver hanwang
[    5.318825] usbcore: registered new interface driver kbtab
[    5.318874] usbcore: registered new interface driver wacom
[    5.656181] tegra-xhci tegra-xhci: new USB bus registered, assigned bus number 9
[    5.677238] usb usb9: New USB device found, idVendor=1d6b, idProduct=0002
[    5.748067] usb usb9: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    5.758789] usb usb9: Product: Nvidia xHCI Host Controller
[    5.767725] usb usb9: Manufacturer: Linux 3.10.96-tegra tegra-xhci
[    5.777365] usb usb9: SerialNumber: tegra-xhci
[    5.785578] hub 9-0:1.0: USB hub found
[    5.857231] tegra-xhci tegra-xhci: new USB bus registered, assigned bus number 10
[    5.868300] usb usb10: New USB device found, idVendor=1d6b, idProduct=0003
[    5.878713] usb usb10: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    5.889515] usb usb10: Product: Nvidia xHCI Host Controller
[    5.898577] usb usb10: Manufacturer: Linux 3.10.96-tegra tegra-xhci
[    5.908330] usb usb10: SerialNumber: tegra-xhci
[    5.916623] hub 10-0:1.0: USB hub found
[    6.358269] usb 10-1: new SuperSpeed USB device number 2 using tegra-xhci
[    6.389246] usb 10-1: New USB device found, idVendor=0955, idProduct=09ff
[    6.399280] usb 10-1: New USB device strings: Mfr=1, Product=2, SerialNumber=6
[    6.409702] usb 10-1: Product: USB 10/100/1000 LAN
[    6.417701] usb 10-1: Manufacturer: Nvidia
[    6.425046] usb 10-1: SerialNumber: 000001000000
[    6.564375] usb 10-1: reset SuperSpeed USB device number 2 using tegra-xhci
[    6.564569] usbcore: registered new interface driver uvcvideo
[    6.564571] USB Video Class driver (1.1.1)
[    7.237917] mmc0: SDHCI controller on sdhci-tegra.3 [sdhci-tegra.3] using ADMA
[    7.377917] mmc1: SDHCI controller on sdhci-tegra.1 [sdhci-tegra.1] using ADMA
[    7.527923] mmc2: SDHCI controller on sdhci-tegra.0 [sdhci-tegra.0] using ADMA
[    7.569986] usbcore: registered new interface driver usbhid
[    7.579621] usbhid: USB HID core driver
[    8.209511] usbcore: registered new interface driver snd-usb-audio
[    8.824721] tegra210-ape-admaif tegra210-admaif: ASoC: Failed to create codec debugfs directory
[    8.881596] tegra-snd-t210ref-mobile-rt565x sound.27:  ADMAIF1 <-> ADMAIF1 mapping ok
[    8.893783] tegra-snd-t210ref-mobile-rt565x sound.27:  ADMAIF2 <-> ADMAIF2 mapping ok
[    8.905768] tegra-snd-t210ref-mobile-rt565x sound.27:  ADMAIF3 <-> ADMAIF3 mapping ok
[    8.917621] tegra-snd-t210ref-mobile-rt565x sound.27:  ADMAIF4 <-> ADMAIF4 mapping ok
[    8.929438] tegra-snd-t210ref-mobile-rt565x sound.27:  ADMAIF5 <-> ADMAIF5 mapping ok
[    8.941193] tegra-snd-t210ref-mobile-rt565x sound.27:  ADMAIF6 <-> ADMAIF6 mapping ok
[    8.952964] tegra-snd-t210ref-mobile-rt565x sound.27:  ADMAIF7 <-> ADMAIF7 mapping ok
[    8.964707] tegra-snd-t210ref-mobile-rt565x sound.27:  ADMAIF8 <-> ADMAIF8 mapping ok
[    8.976462] tegra-snd-t210ref-mobile-rt565x sound.27:  ADMAIF9 <-> ADMAIF9 mapping ok
[    8.988223] tegra-snd-t210ref-mobile-rt565x sound.27:  ADMAIF10 <-> ADMAIF10 mapping ok
[    9.804826] usb-vbus3: disabling
[    9.811877] usb-vbus1: disabling
[  293.889925] usb 9-3: new high-speed USB device number 3 using tegra-xhci
[  293.912262] usb 9-3: New USB device found, idVendor=1a40, idProduct=0101
[  293.912378] usb 9-3: New USB device strings: Mfr=0, Product=1, SerialNumber=0
[  293.912463] usb 9-3: Product: USB 2.0 Hub
[  293.917481] hub 9-3:1.0: USB hub found
[  294.229568] usb 9-3.2: new low-speed USB device number 4 using tegra-xhci
[  294.271621] usb 9-3.2: New USB device found, idVendor=03f0, idProduct=034a
[  294.271659] usb 9-3.2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[  294.271687] usb 9-3.2: Product: HP Elite USB Keyboard
[  294.271710] usb 9-3.2: Manufacturer: Chicony
[  294.272834] usb 9-3.2: ep 0x81 - rounding interval to 64 microframes, ep desc says 80 microframes
[  294.272952] usb 9-3.2: ep 0x82 - rounding interval to 64 microframes, ep desc says 80 microframes
[  294.291165] input: Chicony HP Elite USB Keyboard as /devices/platform/tegra-xhci/usb9/9-3/9-3.2/9-3.2:1.0/input/input2
[  294.292104] hid-generic 0003:03F0:034A.0001: input,hidraw0: USB HID v1.10 Keyboard [Chicony HP Elite USB Keyboard] on usb-tegra-xhc                                            i-3.2/input0
[  294.303798] input: Chicony HP Elite USB Keyboard as /devices/platform/tegra-xhci/usb9/9-3/9-3.2/9-3.2:1.1/input/input3
[  294.304816] hid-generic 0003:03F0:034A.0002: input,hidraw1: USB HID v1.10 Device [Chicony HP Elite USB Keyboard] on usb-tegra-xhci-                                            3.2/input1
[  294.409365] usb 9-3.4: new full-speed USB device number 5 using tegra-xhci
[  294.441434] usb 9-3.4: New USB device found, idVendor=064f, idProduct=2af9
[  294.441445] usb 9-3.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[  294.441451] usb 9-3.4: Product: CodeMeter-Stick
[  294.441456] usb 9-3.4: Manufacturer: WIBU-SYSTEMS AG
[  294.441461] usb 9-3.4: SerialNumber: 000003668997
[  294.441756] usb 9-3.4: ep 0x81 - rounding interval to 1024 microframes, ep desc says 2000 microframes
[  294.477406] hid-generic 0003:064F:2AF9.0003: hiddev0,hidraw2: USB HID v1.11 Device [WIBU-SYSTEMS AG CodeMeter-Stick] on usb-tegra-x                                            hci-3.4/input0
[  294.720209] usb 9-3.1: new full-speed USB device number 6 using tegra-xhci
[  294.747408] usb 9-3.1: New USB device found, idVendor=062a, idProduct=4102
[  294.747526] usb 9-3.1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[  294.747612] usb 9-3.1: Product: 2.4G Wireless Mouse
[  294.747686] usb 9-3.1: Manufacturer: MOSART Semi.
[  294.766854] input: MOSART Semi. 2.4G Wireless Mouse as /devices/platform/tegra-xhci/usb9/9-3/9-3.1/9-3.1:1.0/input/input4
[  294.768822] hid-generic 0003:062A:4102.0004: input,hiddev0,hidraw3: USB HID v1.10 Mouse [MOSART Semi. 2.4G Wireless Mouse] on usb-t                                            egra-xhci-3.1/input0
[  667.985074] usb 9-3: USB disconnect, device number 3
[  667.985113] usb 9-3.1: USB disconnect, device number 6
[  668.050835] usb 9-3.2: USB disconnect, device number 4
[  668.429742] usb 9-3.4: USB disconnect, device number 5
[ 1457.099162] usb 9-3: new low-speed USB device number 7 using tegra-xhci
[ 1457.124246] usb 9-3: New USB device found, idVendor=192f, idProduct=0416
[ 1457.124280] usb 9-3: New USB device strings: Mfr=0, Product=2, SerialNumber=0
[ 1457.124309] usb 9-3: Product: USB Optical Mouse
[ 1457.124877] usb 9-3: ep 0x81 - rounding interval to 64 microframes, ep desc says 80 microframes
[ 1457.133077] input: USB Optical Mouse as /devices/platform/tegra-xhci/usb9/9-3/9-3:1.0/input/input5
[ 1457.133726] hid-generic 0003:192F:0416.0005: input,hidraw0: USB HID v1.11 Mouse [USB Optical Mouse] on usb-tegra-xhci-3/input0
[ 1541.949002] usb 9-3: USB disconnect, device number 7
[ 1546.169334] usb 9-3: new high-speed USB device number 8 using tegra-xhci
[ 1546.191078] usb 9-3: New USB device found, idVendor=1a40, idProduct=0101
[ 1546.191112] usb 9-3: New USB device strings: Mfr=0, Product=1, SerialNumber=0
[ 1546.191120] usb 9-3: Product: USB 2.0 Hub
[ 1546.192290] hub 9-3:1.0: USB hub found
[ 1546.489255] usb 9-3.2: new low-speed USB device number 9 using tegra-xhci
[ 1546.530173] usb 9-3.2: New USB device found, idVendor=03f0, idProduct=034a
[ 1546.530200] usb 9-3.2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 1546.530206] usb 9-3.2: Product: HP Elite USB Keyboard
[ 1546.530211] usb 9-3.2: Manufacturer: Chicony
[ 1546.530567] usb 9-3.2: ep 0x81 - rounding interval to 64 microframes, ep desc says 80 microframes
[ 1546.530579] usb 9-3.2: ep 0x82 - rounding interval to 64 microframes, ep desc says 80 microframes
[ 1546.541117] input: Chicony HP Elite USB Keyboard as /devices/platform/tegra-xhci/usb9/9-3/9-3.2/9-3.2:1.0/input/input6
[ 1546.541347] hid-generic 0003:03F0:034A.0006: input,hidraw0: USB HID v1.10 Keyboard [Chicony HP Elite USB Keyboard] on usb-tegra-xhc                                            i-3.2/input0
[ 1546.552803] input: Chicony HP Elite USB Keyboard as /devices/platform/tegra-xhci/usb9/9-3/9-3.2/9-3.2:1.1/input/input7
[ 1546.553289] hid-generic 0003:03F0:034A.0007: input,hidraw1: USB HID v1.10 Device [Chicony HP Elite USB Keyboard] on usb-tegra-xhci-                                            3.2/input1
[ 1546.659217] usb 9-3.4: new full-speed USB device number 10 using tegra-xhci
[ 1546.691066] usb 9-3.4: New USB device found, idVendor=064f, idProduct=2af9
[ 1546.691075] usb 9-3.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1546.691080] usb 9-3.4: Product: CodeMeter-Stick
[ 1546.691084] usb 9-3.4: Manufacturer: WIBU-SYSTEMS AG
[ 1546.691088] usb 9-3.4: SerialNumber: 000003668997
[ 1546.691289] usb 9-3.4: ep 0x81 - rounding interval to 1024 microframes, ep desc says 2000 microframes
[ 1546.719030] hid-generic 0003:064F:2AF9.0008: hiddev0,hidraw2: USB HID v1.11 Device [WIBU-SYSTEMS AG CodeMeter-Stick] on usb-tegra-x                                            hci-3.4/input0
[ 1546.979993] usb 9-3.1: new full-speed USB device number 11 using tegra-xhci
[ 1547.002415] usb 9-3.1: New USB device found, idVendor=062a, idProduct=4102
[ 1547.002434] usb 9-3.1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 1547.002448] usb 9-3.1: Product: 2.4G Wireless Mouse
[ 1547.002460] usb 9-3.1: Manufacturer: MOSART Semi.
[ 1547.005868] input: MOSART Semi. 2.4G Wireless Mouse as /devices/platform/tegra-xhci/usb9/9-3/9-3.1/9-3.1:1.0/input/input8
[ 1547.006400] hid-generic 0003:062A:4102.0009: input,hiddev0,hidraw3: USB HID v1.10 Mouse [MOSART Semi. 2.4G Wireless Mouse] on usb-t                                            egra-xhci-3.1/input0
[ 1902.468456] usb 9-3.1: USB disconnect, device number 11
[ 2035.095329] usb 9-3: USB disconnect, device number 8
[ 2035.095365] usb 9-3.2: USB disconnect, device number 9
[ 2035.449771] usb 9-3.4: USB disconnect, device number 10
[12646.178814] sdhci: ADMA Err[0x054]: 0x00000000 | ADMA Ptr[0x058]: 0xfdd28010

[linuxdev]

This is relevant:

[    3.152933]  (null): coherent DMA mask is unset

That message comes from “arch/arm64/mm/dma-mapping.c” with this comment:

/*
 * Sanity check the DMA mask - it must be non-zero, and
 * must be able to be satisfied by a DMA allocation.
 */
if (mask == 0) {
          dev_dbg(dev, "coherent DMA mask is unset\n");
            return 0;
}

So something is wrong with DMA. Unfortunately I don’t know where the mask is from (I haven’t researched it), but you’re on the right track. Someone who knows the driver will need to comment…it may be a case of adding some printk’s to find out what went wrong on the mask.

[vidyas]

The fault address i.e. 0xfdd0e000 looks like a proper physical address, but, not sure why SMMU is faulting for this address.
Are kernel and DT from the same release here?

[pejman.habashi]

Thanks for all the helps. I have searched in the forum for DMA problems, and apparently there was a bug in DMA access in L4T R24.2.1, the version that I was using. Here is more details about it:

https://devtalk.nvidia.com/default/topic/967515/l4t-r24-2-pcie-iommu-not-working-when-switches-bridges-are-present/

I had to re-flash the TX1 with Jetpack 3.1 which has the L4T R28.1 and now the DMA problem is resolved.