I have an FPGA module with PCIe IP (x4). When I connect it directly to the TX2, it is working fine. But when I am trying to connect it through a switch (Tried with IDT and Microsemi Switches), it is not getting detected. The switches also are not getting detected in lspci. The same configuration is working fine on x86.
DMESG shows
tegra-pcie 10003000.pcie-controller: link 0 down, retrying
tegra-pcie 10003000.pcie-controller: link 0 down, ignoring
when connecting through IDT switch.
When connected directly, it shows
tegra-pcie 10003000.pcie-controller: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [mem 0x50100000-0x6fffffff]
pci_bus 0000:00: root bus resource [mem 0x70000000-0x7fffffff pref]
pci_bus 0000:00: root bus resource [bus 00-ff]
pci_bus 0000:00: root bus resource [io 0x1000-0xffff]
pci 0000:00:01.0: [10de:10e5] type 01 class 0x060400
pci 0000:00:01.0: PME# supported from D0 D1 D2 D3hot D3cold
iommu: Adding device 0000:00:01.0 to group 55
pci 0000:00:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:00.0: [10ee:9024] type 00 class 0x120000
pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x000fffff 64bit]
pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x000fffff 64bit]
pci 0000:01:00.0: reg 0x20: [mem 0x00000000-0x00000fff]
pci 0000:01:00.0: supports D1
pci 0000:01:00.0: PME# supported from D0 D1 D3hot D3cold
iommu: Adding device 0000:01:00.0 to group 56
pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
pci 0000:00:01.0: BAR 8: assigned [mem 0x50100000-0x503fffff]
pci 0000:01:00.0: BAR 0: assigned [mem 0x50100000-0x501fffff 64bit]
pci 0000:01:00.0: BAR 2: assigned [mem 0x50200000-0x502fffff 64bit]
pci 0000:01:00.0: BAR 4: assigned [mem 0x50300000-0x50300fff]
pci 0000:00:01.0: PCI bridge to [bus 01]
pci 0000:00:01.0: bridge window [mem 0x50100000-0x503fffff]
pcieport 0000:00:01.0: enabling device (0000 -> 0002)
pcieport 0000:00:01.0: Signaling PME through PCIe PME interrupt
pci 0000:01:00.0: Signaling PME through PCIe PME interrupt
pcie_pme 0000:00:01.0:pcie01: service driver pcie_pme loaded
aer 0000:00:01.0:pcie02: service driver aer loaded
tegra-pcie 10003000.pcie-controller: speed change : Gen-1 -> Gen-2
Why is this happening?
Edit:- Connected the same PCIe Card with the same configuration in Connecttech Rudy (Connected to the MiniPCIe port through x1 converter). The switch with all downstream ports and the FPGA EP are getting detected on it.
I tried again with a new switch (Microsemi) on Jetson and Connecttech Rudy. The jetson is giving the following error (I patched the pci-tegra.c to enable manual enumeration as in https://devtalk.nvidia.com/default/topic/997845/jetson-tx1/force-rescan-of-pcie-bus-/) while enumerating. I am getting the same error with the IDT switch as well, with the same exact error codes.
pci_bus 0000:01: busn_res: [bus 01] end is updated to 01
pci_bus 0000:02: busn_res: [bus 02] end is updated to 02
pcieport 0000:00:03.0: AER: Uncorrected (Non-Fatal) error received: id=0060
pcieport 0000:00:03.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0018(Requester ID)
pcieport 0000:00:03.0: device [10de:10e6] error status/mask=00004000/00000000
pcieport 0000:00:03.0: [14] Completion Timeout (First)
pcieport 0000:00:03.0: broadcast error_detected message
pcieport 0000:00:03.0: broadcast mmio_enabled message
pcieport 0000:00:03.0: broadcast resume message
pcieport 0000:00:03.0: AER: Device recovery successful
In Connecttech Rudy, it is getting enumerated correctly and are showing in lspci.
hello @vinuchandran,
I’m working with connect an FPGA module with Jetson TX2 via PCIe, however, I’m facing the same problem with you for end point not getting detected!
Could you please share to me how can I debugging this issue?
as below is my error:
tony@dovanhuong:~$ dmesg | grep pci
[ 0.488359] iommu: Adding device 10003000.pcie-controller to group 49
[ 0.488386] arm-smmu: forcing sodev map for 10003000.pcie-controller
[ 1.141635] tegra-pcie 10003000.pcie-controller: 4x1, 1x1 configuration
[ 1.142814] tegra-pcie 10003000.pcie-controller: PCIE: Enable power rails
[ 1.143256] tegra-pcie 10003000.pcie-controller: probing port 0, using 4 lanes
[ 1.147004] tegra-pcie 10003000.pcie-controller: probing port 2, using 1 lanes
[ 1.574753] tegra-pcie 10003000.pcie-controller: link 0 down, retrying
[ 1.983000] tegra-pcie 10003000.pcie-controller: link 0 down, retrying
[ 2.390663] tegra-pcie 10003000.pcie-controller: link 0 down, retrying
[ 2.392701] tegra-pcie 10003000.pcie-controller: link 0 down, ignoring
[ 2.799308] tegra-pcie 10003000.pcie-controller: link 2 down, retrying
[ 3.207451] tegra-pcie 10003000.pcie-controller: link 2 down, retrying
[ 3.649205] tegra-pcie 10003000.pcie-controller: link 2 down, retrying
[ 3.655800] tegra-pcie 10003000.pcie-controller: link 2 down, ignoring
[ 3.690028] ehci-pci: EHCI PCI platform driver
[ 3.690068] ohci-pci: OHCI PCI platform driver
[ 3.878089] tegra-pcie 10003000.pcie-controller: PCIE: no end points detected
[ 3.892630] tegra-pcie 10003000.pcie-controller: PCIE: Disable power rails
tony@dovanhuong:~$ head -n 1 /etc/nv_tegra_release
R32 (release), REVISION: 3.1, GCID: 18186506, BOARD: t186ref, EABI: aarch64, DATE: Tue Dec 10 07:03:07 UTC 2019
tony@dovanhuong:~$
if you have any help will be higly appreciate,
Thank you in advance