PCIe C5 - I210 card connected on Lane 2 and 4 not detected in Jetson AGX Orin

Hi,

We have 4 I210 connected on C5 through the bridge. But we are able to see only 2 out of that on lspci.
I210 connected on Lane 6 and 7 are detected whereas I210 connected on Lane 2 and 4 are not detected.

We also followed the debug steps for PCIe Link-up failure which is mentioned in the document “JetsonModuleAdaptationAndBringUp”. But still facing the same issue.

Attached the log of “sudo lspci -vvv” and “sudo dmesg | grep pcie

lspci
0000:00:00.0 PCI bridge: NVIDIA Corporation Device 229c (rev a1)
0000:01:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)
0001:00:00.0 PCI bridge: NVIDIA Corporation Device 229e (rev a1)
0001:01:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)
0005:00:00.0 PCI bridge: NVIDIA Corporation Device 229a (rev a1)
0005:01:00.0 PCI bridge: PLX Technology, Inc. PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch (rev ab)
0005:02:01.0 PCI bridge: PLX Technology, Inc. PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch (rev ab)
0005:02:02.0 PCI bridge: PLX Technology, Inc. PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch (rev ab)
0005:02:04.0 PCI bridge: PLX Technology, Inc. PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch (rev ab)
0005:02:05.0 PCI bridge: PLX Technology, Inc. PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch (rev ab)
0005:02:06.0 PCI bridge: PLX Technology, Inc. PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch (rev ab)
0005:02:07.0 PCI bridge: PLX Technology, Inc. PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch (rev ab)
0005:06:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)
0005:07:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)

Configuration from the file → kernel_tegra234-p3701-0004-p3737-0000.dtb
pcie@141a0000 {
compatible = “nvidia,tegra234-pcie\0snps,dw-pcie”;
power-domains = <0x02 0x05>;
reg = <0x00 0x141a0000 0x00 0x20000 0x00 0x3a000000 0x00 0x40000 0x00 0x3a040000 0x00 0x40000 0x00 0x3a080000 0x00 0x40000 0x2b 0x30000000 0x00 0x10000000>;
reg-names = “appl\0config\0atu_dma\0dbi\0ecam”;
status = “okay”;
#address-cells = <0x03>;
#size-cells = <0x02>;
device_type = “pci”;
num-lanes = <0x08>;
num-viewport = <0x08>;
linux,pci-domain = <0x05>;
clocks = <0x02 0xe1 0x02 0xea>;
clock-names = “core\0core_m”;
resets = <0x02 0x82 0x02 0x81>;
reset-names = “apb\0core”;
interrupts = <0x00 0x35 0x04 0x00 0x36 0x04>;
interrupt-names = “intr\0msi”;
interconnects = <0x44 0xe2 0x44 0xe3>;
interconnect-names = “dma-mem\0dma-mem”;
iommus = <0x03 0x14>;
iommu-map = <0x00 0x03 0x14 0x1000>;
msi-parent = <0x35 0x14>;
msi-map = <0x00 0x35 0x14 0x1000>;
dma-coherent;
iommu-map-mask = <0x00>;
#interrupt-cells = <0x01>;
interrupt-map-mask = <0x00 0x00 0x00 0x00>;
interrupt-map = <0x00 0x00 0x00 0x00 0x01 0x00 0x35 0x04>;
nvidia,dvfs-tbl = <0xc28cb00 0xc28cb00 0xc28cb00 0xc28cb00 0xc28cb00 0xc28cb00 0xc28cb00 0x27ac4000 0xc28cb00 0xc28cb00 0x27ac4000 0x5f5e1000 0xc28cb00 0x27ac4000 0x5f5e1000 0x7f22ff40>;
nvidia,max-speed = <0x04>;
nvidia,disable-aspm-states = <0x0f>;
nvidia,controller-id = <0x02 0x05>;
nvidia,tsa-config = <0x200b004>;
nvidia,disable-l1-cpm;
nvidia,aux-clk-freq = <0x13>;
nvidia,preset-init = <0x05>;
nvidia,aspm-cmrt = <0x3c>;
nvidia,aspm-pwr-on-t = <0x14>;
nvidia,aspm-l0s-entrance-latency = <0x03>;
nvidia,bpmp = <0x02 0x05>;
nvidia,aspm-cmrt-us = <0x3c>;
nvidia,aspm-pwr-on-t-us = <0x14>;
nvidia,aspm-l0s-entrance-latency-us = <0x03>;
bus-range = <0x00 0xff>;
ranges = <0x81000000 0x00 0x3a100000 0x00 0x3a100000 0x00 0x100000 0x82000000 0x00 0x40000000 0x2b 0x28000000 0x00 0x8000000 0xc3000000 0x27 0x40000000 0x27 0x40000000 0x03 0xe8000000>;
nvidia,cfg-link-cap-l1sub = <0x1c4>;
nvidia,cap-pl16g-status = <0x174>;
nvidia,cap-pl16g-cap-off = <0x188>;
nvidia,event-cntr-ctrl = <0x1d8>;
nvidia,event-cntr-data = <0x1dc>;
nvidia,dl-feature-cap = <0x30c>;
nvidia,ptm-cap-off = <0x318>;
vddio-pex-ctl-supply = <0x36>;
phys = <0x37 0x38 0x39 0x3a 0x3b 0x3c 0x3d 0x3e>;
phy-names = “p2u-0\0p2u-1\0p2u-2\0p2u-3\0p2u-4\0p2u-5\0p2u-6\0p2u-7”;
vpcie3v3-supply = <0x3f>;
vpcie12v-supply = <0x40>;
phandle = <0x377>;
};
dmesg.txt (5.4 KB)
lspci.txt (76.8 KB)

Not sure what does that mean lane2 and lane4 are not detected. If you are talking some I210 could be detected on the PCIe switch on some lanes and not able to be detected on the reset of lanes, then please check with the switch vendor.

There is nothing to configure on C5.