Hi,
I am working with NVIDIA AGX Xavier Development kit to bringup a PCIe WiFi card on the external PCIe slot.
The WiFi module is a miniPCIe module that supports GEN3 and I use a PCIe to miniPCIe adapter to connect to the external PCIe slot on the AGX Xavier.
Jetson SDK Release: Jetson Linux 35.4.1
Kernel Version: 5.10
I observe some issues with link initialisation and enumeration.
I notice that upon booting, only sometimes the PCIe link is up properly and I can find my device from lspci.
If I issue lspci -vvv sometimes the information is corrupted and there seems to be signal integrity possibly due to the miniPCIe adapter. So I downgraded to GEN2 or GEN1 and I see the information properly. I’m okay with GEN2 and not so greedy to get GEN3 to work for now, but this solution is okay only if the link is up at boot-up.
About 70% of the time, there is no proper PCIe link established during boot. I suspect the link training could have failed during enumeration and didn’t fallback to GEN2 or GEN1 as well which it should have. Sometimes the PCIe controller ‘pcie@141a0000’ also doesn’t show up in lspci when the issue happens.
I searched this forum for some posts and found the following link that suggests the “nvidia,max-speed” DT parameter of ‘pcie@141a0000’ node to be set to 1 in “hardware/nvidia/soc/t19x/kernel-dts/tegra194-soc/tegra194-soc-pcie.dtsi”.
Post: Pcie device can not be recongnized - #4 by megatron
I tried to set “nvidia,max-speed” and also added “nvidia,init-speed” DT parameters and set to 1 in the ‘pcie@141a0000’ node, compiled and loaded the DTB to the target at /boot/dtb and it doesn’t have any effect. I wanted to ensure that these parameters in the DT is being parsed in the first place. It looks like the PCIe tegra driver under “drivers/pci/controller/dwc/pcie-tegra194.c” doesn’t even parse these parameters.
Under the overlay, I found another PCIe driver “/drivers/pci/dwc/pcie-tegra.c” which seems to be parsing the DT node for the above entries but this driver is not even getting compiled or even listed as this driver is compiled only if CONFIG_ARCH_TEGRA_19x_SOC is set. In my case, my .config has CONFIG_ARCH_TEGRA_19x_SOC not set. So that explains it.
If I use an M.2 to miniPCIe adapter and plug it to the M.2 slot underneath the Dev Kit the link is up at GEN3 and there’s no issue observed in PCIe enumeration. I’ve tried it multiple times and everytime the enumeration is proper at boot-up. But the M.2 to miniPCIe adapter kind of hangs upside down and it is inconvenient to make certain tests this way.
I am attaching various logs that I have collected - dmesg, lspci and setpci.
Please check and let me know if more information is needed.
Questions:
-
Can someone clarify which PCIe driver should I use to begin with? Pointers to any forum post/documentation or a bit of explanation would be nice.
-
I would like the PCIe controller driver (for the external PCIe slot) to limit the max speed to 2 so that I improve the chances of getting the link up at boot time. I would like the PCIe controller to start with GEN 2 during the enumeration. How do I accomplish that with the existing driver within the kernel? The DT parameter would be nice but I can even patch the existing driver to hard code the max-speed, I don’t really mind.
-
I tried PCI rescan but didn’t help. Is PCIe hotplug supported?
Attachments:
External PCIe slot + GEN3 Mini PCIe over PCIe to Mini PCIe adapter:
Not working:
lspci_not_working2.txt (12.4 KB)
dmesg_not_working2.txt (73.2 KB)
lspci_not_working.txt (7.7 KB)
dmesg_not_working.txt (74.3 KB)
Working:
lspci_working.txt (12.7 KB)
dmesg_working.txt (72.7 KB)
setpci_gen2.txt (466 Bytes)
lspci_working_setpci_gen2.txt (14.5 KB)
M.2 Slot + GEN3 Mini PCIe over M.2 to MiniPCIe adapter:
lspci_working_m2_slot.txt (15.7 KB)
dmesg_working_m2_slot.txt (72.8 KB)
External PCIe slot + GEN1 Mini PCIe over PCIe to Mini PCIe adapter:
lspci_working_external_pcie_slot_gen1_card.txt (23.7 KB)
dmesg_working_external_pci_slot_gen1_card.txt (76.5 KB)
Thanks,
Sundar