PCIe x1, PCIe x4, link fail at PCIe x16

Hi,

Our carrier board has a PCIe x16 slot and was the same designed as p2822 demo carrier board.
The differences between our carrier board and demo board as below:
1. We didn't use NVHS0_SLVS_REFCLK0_N and NVHS0_SLVS_REFCLK0_P.
2. We didn't use A59 & G55 to enable 12v and 3v3.
3. We didn't use E59 as input source for TS3USB30E UQFN10 COMMON.

We encountered a problem while using PCIe x1 or PCIe x4 device at PCIe x16 slot, it didn't work, but if we use PCI x16 device, it works. 

Please give us some advice to solve this problem, thank you.

Hi, so what is connected to REFCLK pins of x16 slot? Is it Root or Endpoint mode?

And what’s the result of on x16 slot of dev kit with your x1, x4 and x16 devices?

Hi,

thanks for replying, I removed the line 『include “tegra194-p2888-p2822-pcie-plugin-manager.dtsi”』 at tegra194-plugin-manager-p2888-0000.dtsi and set pcie@141a0000 { status = “okay” } only. I guess it’s root mode right ?

Althrough I didn’t include tegra194-p2888-p2822-pcie-plugin-manager.dtsi, I can still find the file at /sys/firmware/devicetree/base/chosen/plugin-manager/odm-data/enable-nvhs-uphy-pcie-c5 and /sys/firmware/devicetree/base/chosen/plugin-manager/odm-data/disable-pcie-c5-endpoint, I can’t figure out why.

If I plug PCIe x1 or PCIe x4 device at PCIe x16 slot, there has no info at lspci and dmesg shows 141a0000.pcie: PCIe link is not up. If I plug PCIe x16 device at PCIe x16 slot, I can find it at lspci and dmesg shows 141a0000.pcie: link is up

thanks

Hello,

 I also found the same problem at PCIe x4 slot, if I use PCIe x4 device at PCIe x4 slot, it can work. But if I use PCIe x1 device at PCIe x4 slot, it didn't work.

 Our PCIe x4 slot used pcie@1418000 (PCIE_C0_CTL) and E11 for PCIE_CLKEREQ_N, D10 for PCIE_RST0_N, E15 for PCIE_REF0_CLKN and E14 for PCIE_REF0_CLKP.

thanks

Please use below patch for the x16 slot first.

---

diff --git a/common/tegra194-p2888-0000-a00.dtsi b/common/tegra194-p2888-0000-a00.dtsi
index 6bbb033..e5771e6 100644
--- a/common/tegra194-p2888-0000-a00.dtsi
+++ b/common/tegra194-p2888-0000-a00.dtsi
@@ -86,7 +86,7 @@
 				 TEGRA194_MAIN_GPIO(A, 1) GPIO_ACTIVE_LOW>;
 			label = "pcie-3v3-reg", "pcie-12v-reg";
 			output-high;
-			status = "disabled";
+			status = "okay";
 		};
 	};
 
@@ -108,10 +108,10 @@
 		status = "okay";
 
 		vddio-pex-ctl-supply = <&p2888_spmic_sd3>;
-		nvidia,disable-aspm-states = <0xf>;
+		nvidia,disable-aspm-states = <0xc>;
 		nvidia,enable-power-down;
 
-		nvidia,max-speed = <1>;
+		nvidia,max-speed = <2>;
 
 		phys = <&p2u_0>;
 		phy-names = "pcie-p2u-0";
@@ -137,6 +137,12 @@
 		nvidia,disable-aspm-states = <0xf>;
 		nvidia,enable-power-down;
 
+		nvidia,plat-gpios =
+			<&tegra_main_gpio TEGRA194_MAIN_GPIO(Z, 2) GPIO_ACTIVE_HIGH    /* 3V3 */
+			 &tegra_main_gpio TEGRA194_MAIN_GPIO(A, 1) GPIO_ACTIVE_LOW     /* 12V */
+		       /*&tegra_main_gpio TEGRA194_MAIN_GPIO(Y, 4) GPIO_ACTIVE_HIGH */ /* I2C */
+			>;
+
 		phys = <&p2u_12>,
 		       <&p2u_13>,
 		       <&p2u_14>,
diff --git a/common/tegra194-p2888-p2822-pcie-plugin-manager.dtsi b/common/tegra194-p2888-p2822-pcie-plugin-manager.dtsi
index 2033df6..e131371 100644
--- a/common/tegra194-p2888-p2822-pcie-plugin-manager.dtsi
+++ b/common/tegra194-p2888-p2822-pcie-plugin-manager.dtsi
@@ -15,38 +15,34 @@
 / {
 	plugin-manager {
 		fragment-pcie-c1-rp {
-			ids = ">=2822-0000-200";
+			ids = "<2822-0000-200";
 			override@0 {
 				target = <&{/pcie@14100000}>;
 				_overlay_ {
-					nvidia,max-speed = <2>;
+					nvidia,max-speed = <1>;
 				};
 			};
 		};
 
-		fragment-pcie-p2822-B00 {
-			ids = ">=2822-0000-400";
+		fragment-pcie-older-than-p2822-B00 {
+			ids = "<2822-0000-400";
 			override@0 {
 				target = <&{/pcie@14100000}>;
 				_overlay_ {
-					nvidia,disable-aspm-states = <0xc>;
+					nvidia,disable-aspm-states = <0xf>;
 				};
 			};
 			override@1 {
 				target = <&{/pcie@141a0000}>;
 				_overlay_ {
-					nvidia,plat-gpios =
-						<&tegra_main_gpio TEGRA194_MAIN_GPIO(Z, 2) GPIO_ACTIVE_HIGH    /* 3V3 */
-						 &tegra_main_gpio TEGRA194_MAIN_GPIO(A, 1) GPIO_ACTIVE_LOW     /* 12V */
-					       /*&tegra_main_gpio TEGRA194_MAIN_GPIO(Y, 4) GPIO_ACTIVE_HIGH */ /* I2C */
-						>;
+					delete-target-property = "nvidia,plat-gpios";
 				};
 			};
 			override@2 {
 				target = <&{/gpio@2200000}>;
 				_overlay_ {
 					pcie-reg-enable {
-						status = "okay";
+						status = "disabled";
 					};
 				};
 			};

Hi,

I've tried the patch but the issue remains.

thanks,

Hi evanic.chen,

Just one more question. Are your x1, x4 devices able to be detected on our devkit x16 slot?

Hi,

Those devices (x1/x2/x4/x16) works fine at x16 slot of p2822 carrier board, that's why I need your help.

thanks,

Hi,

Any update here ?

thanks,

Apologies to get to your issue late.
So, what is the reason for not using A59 and G55? Is it because your carrier board has 12v and 3.3v by default? If not, how are you ensuring that the endpoint connected to x16 slot is receiving 12v and 3.3v? The reason why a x16 device is getting enumerated is because your board somehow has 12v going to slot and endpoint comes up with just 12V available? (and all other devices i.e. x1 and x4 devices need 3.3v also available which is not the case here?)
Basically, the point I would like to know is whether the slot has 12v and 3.3v supplies available? It would be really great if these can be measured through a scope during enumeration

Hi vidyas,

I’ve checked 3.3v and 12v are available on our pcie slots.

These power supplies are directly connect to power source and it was available all the time.

We have 2 pcie slots, one is x16 another is x4, our problem is when we used shorter device on the long slot, it wont’s work (e.g. x1/x4/x8 plug in the x16 slots), but if we use the same length of slot size, it will work. (x16 plug in the x16 slot, x4 plug in the x4 slot)

TKS,

In your carrier board, did you make any dependency on power availability with PRSNT pin in the slot?

Hi Vidyas

We left PRSNT pin floating, it same with dev kit design.

Leif

In dev kit design, PRSNT1 is grounded and all PRSNT2 pins are combined and used to control slot supply. When a card is inserted, card shorts both PRSNT1 and PRSNT2 thereby extending PRSNT1’s ground to PRSNT2 which in turn control slot’s supply.
Since you anyway made supplies always available, I think this shouldn’t matter.
BTW, did you try connecting the same x16 which gets enumerated as x4 (with the help of a x4 to x16 riser card … something like https://www.amazon.com/Express-Female-Riser-Mining-Cable/dp/B00CJE0KJ6 )
What about CLKREQ routing to the slot? If there is no CLKREQ routing, can you please add “nvidia,disable-clock-request” to the respective PCIe controller node’s DT entry and try?

Hi Vidays

I use JET-5308 (x16 to x4 extender), it still can’t work normally.
CLKREQ connect to Xavirer pin C8, I need to add “nvidia,disable-clock-request” ?

Leif

If CLKREQ is routed from endpoint device all the way upto root port, then, it should work. Just to rule out issues with CLKREQ, can you please add “nvidia,disable-clock-request” to the device-tree node of the respective PCIe controller?

Our SW add “nvidia,disable-clock-request” then system can’t boot.
The last message of boot is “tegra-pcie-dw 141a0000.pcie: PCIe link is not up…!”.

Can you please attach full log?