Tegra NX PCIe Reference Clock Design

krisrst · October 28, 2020, 6:19pm

Hi!

Does anyone know if the Tegra NX PCIe (root port mode) supports the following:

Spread Spectrum = SSC

Common reference clock without SSC
Common reference clock with SSC
Separate reference clocks with SSC (SRIS) (already answered on these forums here)
Separate reference clocks without SSC (SRNS)

As well as knowing the default reference clock mode, I wish to know how to switch between them. There is some mention on the subject but it would be nice with something more official…

vidyas · October 29, 2020, 5:43am

Tegra NX does support SSC but with only a Common reference clock and also we don’t suggest disabling SSC

krs1 · October 29, 2020, 9:07am

Thanks for quick reply Vidyas. If we want to disable SSC (from boot) for testing purposes, how can it be done? Is the following patch the way to go?

    diff --git a/drivers/clk/tegra/clk-pll.c b/drivers/clk/tegra/clk-pll.c
index 87e7bf9cabb3..ceeac9d9c6e1 100644
--- a/drivers/clk/tegra/clk-pll.c
+++ b/drivers/clk/tegra/clk-pll.c
@@ -452,7 +452,7 @@ static void _clk_pll_disable(struct clk_hw *hw)
 
 static void pll_clk_start_ss(struct tegra_clk_pll *pll)
 {
-       if (pll->params->defaults_set && pll->params->ssc_ctrl_reg) {
+       if (pll->params->defaults_set && pll->params->ssc_ctrl_reg && 0) {
                u32 val = pll_readl(pll->params->ssc_ctrl_reg, pll);
 
                val |= pll->params->ssc_ctrl_en_mask;
@@ -463,7 +463,7 @@ static void pll_clk_start_ss(struct tegra_clk_pll *pll)
 
 static void pll_clk_stop_ss(struct tegra_clk_pll *pll)
 {
-       if (pll->params->defaults_set && pll->params->ssc_ctrl_reg) {
+       if (pll->params->defaults_set && pll->params->ssc_ctrl_reg && 0) {
                u32 val = pll_readl(pll->params->ssc_ctrl_reg, pll);
 
                val &= ~pll->params->ssc_ctrl_en_mask;
@@ -2583,6 +2583,7 @@ static int clk_plle_tegra210_enable(struct clk_hw *hw)
        if (pll->params->ssc_ctrl_reg != PLLE_SS_CTRL)
                goto out;
 
+#if 0
        val = pll_readl(PLLE_SS_CTRL, pll);
        val &= ~(PLLE_SS_CNTL_CENTER | PLLE_SS_CNTL_INVERT);
        val &= ~PLLE_SS_COEFFICIENTS_MASK;
@@ -2594,6 +2595,7 @@ static int clk_plle_tegra210_enable(struct clk_hw *hw)
        val &= ~PLLE_SS_CNTL_INTERP_RESET;
        pll_writel(val, PLLE_SS_CTRL, pll);
        fence_udelay(1, pll->clk_base);
+#endif

krisrst · October 29, 2020, 4:19pm

Umm actually on the NX I see now that the clocks are managed by the dedicated clock processor, which initialises the PLL E by device tree. The forum post here says I can put this into the bpmp-fw-dtb:

clock@plle {
     clk-id = <0x200>;
     pll_freq_table = <0x249f000 0x5f5e100 0x2 0x7d 0x18 0xffffffff 0xffffffff 0xffffffff 0xffffffff>;
};

And flash:

sudo ./flash -r -k bpmp-fw-dtb jetson-tx2 mmcblk0p1

…but

I get a SHA2 signature error on boot, and NX is stuck in boot loop
If I do a full install (flash everything) the bootloop disappears, but SSC is still enabled (measure by oscilloscope)

Can you provide the necessary changes to the bpmp-fw-dtb, as well as instructions to flash with signature?

vidyas · November 3, 2020, 3:32pm

Are you sure “sudo ./flash -r -k bpmp-fw-dtb jetson-tx2 mmcblk0p1” is the flash command you used? I mean since this is NX, shouldn’t you be using “jetson-xavier-nx-devkit” (for SD card based board) and “jetson-xavier-nx-devkit-emmc.” (for emmc based board) instead of ‘jetson-tx2’??

Anyway, NX has two slots
M.2 Key-E slot which is owned by C4 controller and
M.2 Key-M slot which is owned by C5 controller
Both controllers (C4 and C5) have different PLL sources. So, if the spread needs to be disabled for C4 then, please apply the following patch

diff --git a/include/parts/uphy/tegra194-bpmp-p2888-uphy.dtsi b/include/parts/uphy/tegra194-bpmp-p2888-uphy.dtsi
index 7f1a3fd..b0fa711 100644 (file)
 
--- a/include/parts/uphy/tegra194-bpmp-p2888-uphy.dtsi
+++ b/include/parts/uphy/tegra194-bpmp-p2888-uphy.dtsi
@@ -1,4 +1,4 @@
-
+#include <mach-t194/clk-t194.h>
 / {
        uphy {
                status = "okay";
@@ -6,4 +6,11 @@
                ufs-config = "UFS_x1_L1";
                nvhs-owner = "PCIE";
        };
+       clocks {
+                       clock@plle {
+                               clk-id = <TEGRA194_CLK_PLLE>;
+                               /* disable ssc on PLLE */
+                               pll_freq_table = <38400000 100000000 2 125 24 (-1) (-1) (-1) (-1)>;
+                       };
+       };
 };

If it is for the C5 controller, then, use the below patch

diff --git a/include/parts/uphy/tegra194-bpmp-p2888-uphy.dtsi b/include/parts/uphy/tegra194-bpmp-p2888-uphy.dtsi
index 7f1a3fd..b63d02e 100644 (file)
--- a/include/parts/uphy/tegra194-bpmp-p2888-uphy.dtsi
+++ b/include/parts/uphy/tegra194-bpmp-p2888-uphy.dtsi
@@ -1,4 +1,4 @@
-
+#include <mach-t194/clk-t194.h>
 / {
        uphy {
                status = "okay";
@@ -6,4 +6,11 @@
                ufs-config = "UFS_x1_L1";
                nvhs-owner = "PCIE";
        };
+       clocks {
+               clock@pllnvhs {
+                       clk-id = <TEGRA194_CLK_PLLNVHS>;
+                       /* disable ssc on PLLNVHS */
+                       pll_freq_table = <38400000 100000000 2 125 24 (-1) (-1) (-1) (-1)>;
+               };
+       };
 };

krisrst · November 5, 2020, 7:11pm

@vidyas,

Thanks for the reply!

I wrote jetson-tx2 in the flash command as a reference to the similar question for the X2. Sorry for the confusion. I am using the correct command in the lab.

I will try out these modifications tomorrow and report the results.

Again thanks!

krisrst · November 16, 2020, 8:00pm

Hi Vidyas,

We have modified the bpmp device tree to disable SSC. Thank you for this info. We are able to measure with oscilloscope that the spread is gone.

We have now done a simple test in SRNS mode - that is, root port (NX) talks with a generic endpoint without propagating the refclk. This seems to be successful and we are able to communicate with the endpoint.

Sorry to bother you with this, but is it possible for Nvidia to comment whether there is any risk involved in using the Tegra NX in SRNS mode? I assume this is not officially supported on your side.

vidyas · November 17, 2020, 6:26am

Yes. you are correct that this is not officially supported.
Reasons being since the same PLL is used for both USB and PCIe and USB certification needs spread to be enabled, if you don’t care about USB much (i.e. the certification part) and ok to have any EMI issues that might come because of spread being disabled, then, it is ok to keep the spread disabled.

krisrst · November 17, 2020, 7:20am

Again thanks for quick reply - we are OK with spread being disabled -
BUT the actual question is: does Nvidia officially support that the root port (NX) and endpoint work with independent 100 MHz reference clocks?

I.e.
[100 MHz refclk, no SSC] —> NX(RP) <— PCIe cabling without refclk —> ENDPOINT <— [100 MHz refclk, no SSC]

This is basically SRNS. Separate reference clocks without SSC.

krisrst · November 17, 2020, 8:21am

For anyone that needs to do this on their NX kits (or similar):

Vidyas patches are from a git repo we do not have access to. So what you need to do:

Pay attention to the #include directive in the patch. While I cannot find <mach-t194/clk-t194.h>, I can find the clock id - TEGRA194_CLK_PLLNVHS by just grepping through everything after doing source_sync.sh. Use the clock ID from there and mind that you use the same clock ID that is for the L4T version you are using. E.g. source_sync.sh -d <YOUR_L4T_VERSION>, then “find . -type f -exec grep TEGRA194_CLK_PLLNVHS {} ;”

Then you device-tree-decompile the relevant bpmp in your Linux_for_Tegra folder (where flash.sh resides. For example:

If you do sudo ./flash.sh -r jetson-xavier-nx-devkit-emmc mmcblk0p1:
Then look at the jetson-xavier-nx-devkit-emmc.conf
cat jetson-xavier-nx-devkit-emmc.conf | grep source
source ${LDK_DIR}/p3668.conf.common";
And if you look for bpmp in that file you will find the one you are flashing:
cat p3668.conf.common | grep -i bpmp
BPFDTB_FILE=tegra194-a02-bpmp-p3668-a00.dtb;
Look for that file
find . -name tegra194-a02-bpmp-p3668-a00.dtb
./bootloader/t186ref/tegra194-a02-bpmp-p3668-a00.dtb
./bootloader/tegra194-a02-bpmp-p3668-a00.dtb
Then decompile one of these
dtc -o test.dts ./bootloader/t186ref/tegra194-a02-bpmp-p3668-a00.dtb
In test.dts, locate the “clock” parent and insert the patch, using the correct (!) clock id

clocks {
clock@pllnvhs {
clk-id = <0xf3>;
pll_freq_table = <0x249f000 0x5f5e100 0x02 0x7d 0x18 0xffffffff 0xffffffff 0xffffffff 0xffffffff>;
};
Compile back to dtb and copy these back
dtc -o test.dtb test.dts
cp test.dtb ./bootloader/t186ref/tegra194-a02-bpmp-p3668-a00.dtb
cp test.dtb ./bootloader/tegra194-a02-bpmp-p3668-a00.dtb
Noting I needed sudo in front of one of these.
Then flash the entire kit.
sudo ./flash.sh -r jetson-xavier-nx-devkit-emmc mmcblk0p1

Note that we have verified the SSC off using oscilloscope.

vidyas · November 17, 2020, 9:39am

Usually, it should work but to answer your question, Yes, Nvidia doesn’t support this configuration officially.
Please note that it is possible to get AER corrected errors if any of the ASPM states are enabled.

krisrst · November 17, 2020, 9:53am

Thanks a lot for you feedback Vidyas :-)

Topic		Replies	Views
PCIe not being recognized - TX2 Jetson TX2	20	3071	March 16, 2020
Disabling PCIe SSC on Xavier in runtime on Jetpack 4.4 Jetson AGX Xavier pcie	18	1577	May 12, 2021
Disable Spread Spectrum (SSC) on the TX1 with a 28.1 release Jetson TX1	16	2447	October 18, 2021
TX1 cannot successful disable SSC (Spread Spectrum Clocking) PEX_CLK0_P & PEX_CLK0_N. Jetson TX1	5	979	July 4, 2018
Jetson TX1 PCIe X4 slot with extend cable or riser card Jetson TX1	22	4893	October 18, 2021
Disable Spread Spectrum (SSC) on the TX2 with a 28.2 release,but kernel stop Jetson TX2	3	1038	October 18, 2021
Sometimes not booting in Jetson Xavier NX Jetson Xavier NX boot , kernel	34	1104	January 16, 2024
What are best way for disable Jetson TX1 PCIe SSC ? Jetson TX1	5	1348	October 18, 2021
How can I use Freescale‘s sgtl5000 on TX2？ Jetson TX2	9	1688	October 18, 2021
Tegra Chip ID not yet available ???? Jetson TX2	11	2337	February 7, 2018

Tegra NX PCIe Reference Clock Design

Related topics