Need a help to A PCIe to USB2.0 card support

Autonomous Machines Jetson & Embedded Systems Jetson Nano
,
#1

hello, I am using a Jetson nano B01 board and a MCS9990 PCIe-to-USB card to verify if it works, since I am planing to design a customized carrier board with more USB2.0 ports to avoid USB2.0 bandwidth issue.
when connect the MCS9990 board， a standard PCIe card with M2 KeyE converter cable, found below message:

nano@nano-desktop : ~ $ lspci
00:01.0 PCI bridge: NVIDIA Corporation Device 0fae (rev a1)
00:02.0 PCI bridge: NVIDIA Corporation Device 0faf (rev a1)
01:00.0 USB controller: MosChip Semiconductor Technology Ltd. MCS9990 PCIe to 4‐Port USB 2.0 Host Controller
01:00.1 USB controller: MosChip Semiconductor Technology Ltd. MCS9990 PCIe to 4‐Port USB 2.0 Host Controller
01:00.2 USB controller: MosChip Semiconductor Technology Ltd. MCS9990 PCIe to 4‐Port USB 2.0 Host Controller
01:00.3 USB controller: MosChip Semiconductor Technology Ltd. MCS9990 PCIe to 4‐Port USB 2.0 Host Controller
01:00.4 USB controller: MosChip Semiconductor Technology Ltd. MCS9990 PCIe to 4‐Port USB 2.0 Host Controller
01:00.5 USB controller: MosChip Semiconductor Technology Ltd. MCS9990 PCIe to 4‐Port USB 2.0 Host Controller
01:00.6 USB controller: MosChip Semiconductor Technology Ltd. MCS9990 PCIe to 4‐Port USB 2.0 Host Controller
01:00.7 USB controller: MosChip Semiconductor Technology Ltd. MCS9990 PCIe to 4‐Port USB 2.0 Host Controller
02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)

dmsg message:

[ 1.769943] usb 7-1: new low-speed USB device number 2 using ohci-pci
[ 1.769987] mc-err: (0) csr_afir: EMEM address decode error
[ 1.775606] mc-err: status = 0x2000000e; addr = 0x77d97c00
[ 1.781302] mc-err: secure: no, access-type: read, SMMU fault: none
[ 1.788005] tegra_dvfs: vdd-gpu-vts: registered
[ 1.788694] tegra_core_action core_dvfs_cdev_floor: Tegra CORE DVFS ‘floor cooling device’ registered

The USB device attached to the MCS9990 does not work.
any suggestion to fix it？ thx

#2

I can’t be of much help, but when seeing the SMMU fault, it makes me think of a driver issue.

In the 32-bit ARM drivers physical addresses tended to be used, but in 64-bit ARM the SMMU is used for virtual addressing. If one were to compile the driver without SMMU on 64-bit, then it would fail like this due to lack of porting memory address changes to support virtual SMMU-based addressing.

Are you sure you have the right driver for 64-bit?

Note that PCIe function would work correctly if this is the wrong driver, and then the USB side would fail with an SMMU message.

#4

Hi,
Please share the release version for reference( $ head -1 /etc/nv_tegra_release ).

M2 KeyE slot has one PCIe one lane and one USB2.0. So the card is connect to PCIe pins to use the one lane bandwidth? Please also share dmesg of booting for reference.

#6

hi,
thanks for the tip. I am not quite sure if the driver is 64-bit. I tired to turn off the SMMU, and re-compiled, but no luck.
also, I tried another card with uPD720202 (PCIe-to-USB3.0 port, SMMU enabled), this card works. the cameras can be configured, but the total bandwidth of USB2.0 is quite low, I attached four 4K USB2.0 cameras to the card, but only one can work. it tells no enough bandwidth for other new USB cameras：

[ 371.348365] usb 1-2 **: Not enough bandwidth for new device state.
[ 371.354365] usb 1-2: Not enough bandwidth for altsetting 11

#7

Was the dmesg log from when you had SMMU off? Normally it would be on, and I would not suggest turning off the SMMU (though it could be a useful test, the driver or other software really needs to be fixed). This log showed a repeating corrected error, which tells me there is something seriously wrong in drier or software, and this wasn’t just a one-time error (and likely it isn’t a hardware error).

There are many cameras which cannot operate in slower modes, so not enough bandwidth is common when trying to use a USB3 camera on a USB2 slot, or when using two or more cameras on a USB port only capable of feeding one camera. A 4K camera would definitely need a lot of bandwidth, and sharing via a HUB would fail. You would need two different USB root HUBs.

Tip: “lsusb -t” lists in a tree format, and the “480M” at the end of a line shows USB2, while a “5000M” or “10000M” indicate USB3 (gen. 1 and gen. 2, respectively). The root of a device will show “Class=root_hub”, and you would not want two cameras using the same “root_hub”.

#8

Hi Dane
i am using this version : R32 (release), REVISION: 4.4, GCID: 23942405, BOARD: t210ref, EABI: aarch64,
attach the dmsg log:
dmesg.txt (94.6 KB)

#9

thanks for the tips, I check the USB root tree and found uPD720202 chip only support one USB2.0 root. that cause the bandwidth issue.
so I have to back to MCS9990 solution (which has 4 USB2.0 roots).

#10

I’m suspecting that the DMA controller of this particular USB add-on card has the capability to address only 32-bit addresses and since the PCIe controller’s SMMU is disabled in Nano, even if the driver is placing a request to allocate only 32-bit addresses, it may not be getting honored.
Could you please try with the below patch and see if it works? (just look for the PCIe node in the DT and remove the iommu entries that are currently commented out)

diff --git a/kernel-dts/porg-platforms/tegra210-porg-pcie.dtsi b/kernel-dts/porg-platforms/tegra210-porg-pcie.dtsi
index 1b4cd54..7c96529 100644
--- a/kernel-dts/porg-platforms/tegra210-porg-pcie.dtsi
+++ b/kernel-dts/porg-platforms/tegra210-porg-pcie.dtsi
@@ -19,9 +19,6 @@
  */
 / {
        pcie@1003000 {
-               /delete-property/ iommus;
-               /delete-property/ iommu-map;
-               /delete-property/ iommu-map-mask;
                nvidia,wake-gpio = <&gpio TEGRA_GPIO(A, 2) 0>;
                nvidia,pmc-wakeup = <&tegra_pmc
                                PMC_WAKE_TYPE_EVENT 0 PMC_TRIGGER_TYPE_LOW>;
#11

hi Vidyas
I‘ve tried the patch, but not work.
I tried same config but change the Nano to NX, it works… PCIe works, USB ports work, strange…
is there any different between Nano PCIe driver and NX PCIe driver ?..

#14

Well. The PCIe controllers are totally different between Nano and NX.
Did you make sure that the DT change is indeed present on the target?
Can you please share the full log? The one shared on Dec-16th doesn’t have the errors pointed out in the beginning.
Another way to debug this issue is to prints in the places where memory gets allocated and see if the address is same as the one indicated by the mc-err.