no PCIe link with some devices

parafin · October 8, 2018, 8:57am

Hi!

I’m having a problem with PCIe in Jetson AGX Xavier. I’ve tried plugging in several PCIe cards into 8x PCIe slot on devkit and while some of them do appear in lspci, others fail to establish PCIe link. As an example of not working card I can give Samtec PCIEA-4G2-H-01: https://datasheet.octopart.com/PCIEA-4G2-H-01-Samtec-datasheet-97182846.pdf
It works fine with TX1 and TX2 (and ordinary PCs). I’m attaching dmesg and lspci -vvv output from both TX2 and Xavier. Obvious difference is that Xavier supports PCIe version 4, so maybe there is some incompatibility caused by that upgrade.
Is there something you can suggest to try? Are there any configuration options in devicetree or somewhere else?
dmesg-tx2.txt (63.6 KB)
lspci-tx2.txt (9.82 KB)
dmesg-xavier.txt (66.8 KB)
lspci-xavier.txt (6.63 KB)

vidyas · October 9, 2018, 8:45am

Can you please provide more info on the codebase you are using? (I mean the release info etc…)

parafin · October 9, 2018, 8:59am

Hi,

I’m using L4T 31.0.2 for Xavier and 28.2.1 for TX2 (but really it worked with TX1 and TX2 on all L4T versions). Nothing else I can add I think, it’s just ordinary fresh install running on DevKits.

Andrey1984 · October 9, 2018, 8:56pm

PCI-e Optane 905 ssd [4 lanes] doesn’t appear to work with Xavier, though worked with tx2.
At the early beginning led lamps are lighting, but after a dozen of seconds they got powered off as if the device unpowered.

I am using the latest 4.1 release.

karthik.parameswaran · October 15, 2018, 4:42pm

Same phenomenon observed on my Jetson Xavier, devices that get detected on TX2 not getting detected on the Jetson Xavier

sukriti.choudhary · October 16, 2018, 5:18pm

Hi,

I am having the same issue on Jetson Xavier. There is one device that works on both TX2 and Jetson Xavier, while a second device only works on the TX2

vidyas · October 18, 2018, 4:41pm

Can you please apply below patches and check if the issue gets resolved?

cd kernel/nvidia

diff --git a/drivers/pci/host/pcie-tegra-dw.c b/drivers/pci/host/pcie-tegra-dw.c
index fd9de65..a5775da 100644
--- a/drivers/pci/host/pcie-tegra-dw.c
+++ b/drivers/pci/host/pcie-tegra-dw.c
@@ -2682,8 +2682,8 @@ static int tegra_pcie_dw_parse_dt(struct tegra_pcie_dw *pcie)
                                return gpio;

                        f = (flags & OF_GPIO_ACTIVE_LOW) ?
-                           (GPIOF_OUT_INIT_LOW | GPIOF_ACTIVE_LOW) :
-                            GPIOF_OUT_INIT_HIGH;
+                           (GPIOF_OUT_INIT_HIGH | GPIOF_ACTIVE_LOW) :
+                            GPIOF_OUT_INIT_LOW;

                        ret = devm_gpio_request_one(pcie->dev, gpio, f, NULL);
                        if (ret < 0) {

cd hardware/nvidia/platform/t19x/galen/kernel-dts

diff --git a/common/tegra194-p2888-0000-a00.dtsi b/common/tegra194-p2888-0000-a00.dtsi
index 6bbb033..6c71a87 100644
--- a/common/tegra194-p2888-0000-a00.dtsi
+++ b/common/tegra194-p2888-0000-a00.dtsi
@@ -79,17 +79,6 @@
                };
        };

-       gpio@2200000 {
-               pcie-reg-enable {
-                       gpio-hog;
-                       gpios = <TEGRA194_MAIN_GPIO(Z, 2) GPIO_ACTIVE_HIGH
-                                TEGRA194_MAIN_GPIO(A, 1) GPIO_ACTIVE_LOW>;
-                       label = "pcie-3v3-reg", "pcie-12v-reg";
-                       output-high;
-                       status = "disabled";
-               };
-       };
-
        pcie@14180000 {
                status = "okay";

diff --git a/common/tegra194-p2888-p2822-pcie-plugin-manager.dtsi b/common/tegra194-p2888-p2822-pcie-plugin-manager.dtsi
index 4fe63fb..a83742c 100644
--- a/common/tegra194-p2888-p2822-pcie-plugin-manager.dtsi
+++ b/common/tegra194-p2888-p2822-pcie-plugin-manager.dtsi
@@ -52,14 +52,6 @@
                                                >;
                                };
                        };
-                       override@2 {
-                               target = <&{/gpio@2200000}>;
-                               _overlay_ {
-                                       pcie-reg-enable {
-                                               status = "okay";
-                                       };
-                               };
-                       };
                };

                fragment-disable-uphy-rx-idle {

Andrey1984 · October 18, 2018, 4:51pm

Thank you for the update.
It appears, that I shall get to Host PC, to Jetpack folder, then update the targets and then flash Xavier with the updated content, then check if that will resolve the issue.

Andrey1984 · October 19, 2018, 8:45pm

In my case I have dts files rather than dtsi mentioned in the post by vidyas:

/Downloads/sources/public_sources/hardware/nvidia/platform/t19x/galen/kernel-dts$ ls
Makefile  tegra194-p2888-0001-e3366-1199.dts
common    tegra194-p2888-0001-p2822-0000.dts

Otherwise dtsi files are in a subfolder common, though
trying to re-flash with the patched files

Andrey1984 · October 19, 2018, 9:47pm

In my case, I just downloaded sources and they have had already the changes as per the post above, but when I flashed with them the situation hasn’t changed and the pcie 4lanes nvme[not M.2] just blinks for a second and gets off. May be other folks will be more fortunate than me.

parafin · November 1, 2018, 3:54pm

Hi vidyas,

the changes you proposed indeed helped. Patch you posted is reversed though, so in reality it looks like that:

--- kernel/nvidia/drivers/pci/host/pcie-tegra-dw.c.orig 2018-09-29 06:32:13.000000000 +0200
+++ kernel/nvidia/drivers/pci/host/pcie-tegra-dw.c      2018-11-01 15:32:21.657577341 +0100
@@ -2682,8 +2682,8 @@
                                return gpio;
 
                        f = (flags & OF_GPIO_ACTIVE_LOW) ?
-                           (GPIOF_OUT_INIT_HIGH | GPIOF_ACTIVE_LOW) :
-                            GPIOF_OUT_INIT_LOW;
+                           (GPIOF_OUT_INIT_LOW | GPIOF_ACTIVE_LOW) :
+                            GPIOF_OUT_INIT_HIGH;
 
                        ret = devm_gpio_request_one(pcie->dev, gpio, f, NULL);
                        if (ret < 0) {
--- hardware/nvidia/platform/t19x/galen/kernel-dts/common/tegra194-p2888-0000-a00.dtsi.orig     2018-09-29 06:30:53.000000000 +0200
+++ hardware/nvidia/platform/t19x/galen/kernel-dts/common/tegra194-p2888-0000-a00.dtsi  2018-11-01 15:29:46.958631704 +0100
@@ -79,6 +79,17 @@
                };
        };
 
+       gpio@2200000 {
+               pcie-reg-enable {
+                       gpio-hog;
+                       gpios = <TEGRA194_MAIN_GPIO(Z, 2) GPIO_ACTIVE_HIGH
+                                TEGRA194_MAIN_GPIO(A, 1) GPIO_ACTIVE_LOW>;
+                       label = "pcie-3v3-reg", "pcie-12v-reg";
+                       output-high;
+                       status = "disabled";
+               };
+       };
+
        pcie@14180000 {
                status = "okay";
 
--- hardware/nvidia/platform/t19x/galen/kernel-dts/common/tegra194-p2888-p2822-pcie-plugin-manager.dtsi.orig    2018-11-01 15:30:38.908277659 +0100
+++ hardware/nvidia/platform/t19x/galen/kernel-dts/common/tegra194-p2888-p2822-pcie-plugin-manager.dtsi 2018-11-01 15:31:43.647836417 +0100
@@ -52,6 +52,14 @@
                                                >;
                                };
                        };
+                       override@2 {
+                               target = <&{/gpio@2200000}>;
+                               _overlay_ {
+                                       pcie-reg-enable {
+                                               status = "okay";
+                                       };
+                               };
+                       };
                };
 
                fragment-disable-uphy-rx-idle {

I hope these changes and some fix for make modules_prepare (e.g. https://devtalk.nvidia.com/default/topic/1042391/jetson-agx-xavier/is-it-possible-to-build-kernel-module-in-xavier-solved-/post/5288000/#5288000) will be integrated into next L4T release.

vidyas · November 9, 2018, 7:30am

Indeed. Next release is going to have these changes.

Andrey1984 · November 9, 2018, 7:48am

do you mean that the changes that will need be take place on the moment of kernel rebuild is

+                           (GPIOF_OUT_INIT_LOW | GPIOF_ACTIVE_LOW) :
+                            GPIOF_OUT_INIT_HIGH;

or

+                           (GPIOF_OUT_INIT_HIGH | GPIOF_ACTIVE_LOW) :
+                            GPIOF_OUT_INIT_LOW;

I tried with the former and I am just wondering if I shall update them to the latter and try again

vidyas · November 9, 2018, 8:59am

Following is the one to be applied

+                           (GPIOF_OUT_INIT_LOW | GPIOF_ACTIVE_LOW) :
+                            GPIOF_OUT_INIT_HIGH;

BTW, device tree changes also need to be applied

Andrey1984 · November 9, 2018, 9:34am

thank you for the clarification.

Andrey1984 · November 9, 2018, 9:37am

However these changes turned out to be already applied in the source, as far as I am concerned to believe. I mean that if I will download the sources archive and unpack them they will have that values already in place, as it turned out to me.

parafin · November 9, 2018, 7:45pm

You said the same about original reversed patch. Try starting from the scratch and applying a patch I posted previously.

Andrey1984 · November 10, 2018, 7:25am

The issue has been resolved with the latest Jetpack 4.1.1 release. Thanks.
It seems that the voltage was king of triggering low consumption mode in an enforced way that used to disable power supply to ssd that resulted in powering off of the latter. However after the latest patch ssd seems to work fine.

Topic		Replies	Views
PCIe Device - No Link Jetson AGX Xavier	29	4398	September 15, 2021
Jetson Xavier not booting with PCIe x1 Switch Jetson AGX Xavier	12	1323	October 18, 2021
AGX Xavier: No PCIe link Jetson AGX Xavier pcie , kernel , board-design , fpga	15	755	August 28, 2023
PCIe to i210 not working with JetPack 5.1.2 Jetson AGX Xavier pcie	21	511	January 29, 2024
Jetson stuck before boot with black screen when PCI-e device is connected Jetson AGX Xavier pcie	34	431	April 24, 2024
Jetson AGX Xavier Pcie(Root) detection of Device needs jetson reboot Jetson AGX Xavier pcie , fpga	25	2062	July 12, 2023
Trouble getting any PCIE devices to show Jetson AGX Xavier pcie	13	1365	October 18, 2021
NVIDIA AGX Xavier - PCIe link is established randomly Jetson AGX Xavier pcie	3	461	February 19, 2024
Xavier not recognizing PCIe USB 3.1 card Jetson AGX Xavier	7	1616	October 18, 2021
PCIE adapters that require +3.3V not being detected by the system Jetson AGX Xavier	7	1258	September 10, 2020

no PCIe link with some devices

Related topics