Tegra NX PCIe Reference Clock Design

Hi!

Does anyone know if the Tegra NX PCIe (root port mode) supports the following:

Spread Spectrum = SSC

  • Common reference clock without SSC
  • Common reference clock with SSC
  • Separate reference clocks with SSC (SRIS) (already answered on these forums here)
  • Separate reference clocks without SSC (SRNS)

As well as knowing the default reference clock mode, I wish to know how to switch between them. There is some mention on the subject but it would be nice with something more official…

Tegra NX does support SSC but with only a Common reference clock and also we don’t suggest disabling SSC

Thanks for quick reply Vidyas. If we want to disable SSC (from boot) for testing purposes, how can it be done? Is the following patch the way to go?

    diff --git a/drivers/clk/tegra/clk-pll.c b/drivers/clk/tegra/clk-pll.c
index 87e7bf9cabb3..ceeac9d9c6e1 100644
--- a/drivers/clk/tegra/clk-pll.c
+++ b/drivers/clk/tegra/clk-pll.c
@@ -452,7 +452,7 @@ static void _clk_pll_disable(struct clk_hw *hw)
 
 static void pll_clk_start_ss(struct tegra_clk_pll *pll)
 {
-       if (pll->params->defaults_set && pll->params->ssc_ctrl_reg) {
+       if (pll->params->defaults_set && pll->params->ssc_ctrl_reg && 0) {
                u32 val = pll_readl(pll->params->ssc_ctrl_reg, pll);
 
                val |= pll->params->ssc_ctrl_en_mask;
@@ -463,7 +463,7 @@ static void pll_clk_start_ss(struct tegra_clk_pll *pll)
 
 static void pll_clk_stop_ss(struct tegra_clk_pll *pll)
 {
-       if (pll->params->defaults_set && pll->params->ssc_ctrl_reg) {
+       if (pll->params->defaults_set && pll->params->ssc_ctrl_reg && 0) {
                u32 val = pll_readl(pll->params->ssc_ctrl_reg, pll);
 
                val &= ~pll->params->ssc_ctrl_en_mask;
@@ -2583,6 +2583,7 @@ static int clk_plle_tegra210_enable(struct clk_hw *hw)
        if (pll->params->ssc_ctrl_reg != PLLE_SS_CTRL)
                goto out;
 
+#if 0
        val = pll_readl(PLLE_SS_CTRL, pll);
        val &= ~(PLLE_SS_CNTL_CENTER | PLLE_SS_CNTL_INVERT);
        val &= ~PLLE_SS_COEFFICIENTS_MASK;
@@ -2594,6 +2595,7 @@ static int clk_plle_tegra210_enable(struct clk_hw *hw)
        val &= ~PLLE_SS_CNTL_INTERP_RESET;
        pll_writel(val, PLLE_SS_CTRL, pll);
        fence_udelay(1, pll->clk_base);
+#endif

Umm actually on the NX I see now that the clocks are managed by the dedicated clock processor, which initialises the PLL E by device tree. The forum post here says I can put this into the bpmp-fw-dtb:

clock@plle {
     clk-id = <0x200>;
     pll_freq_table = <0x249f000 0x5f5e100 0x2 0x7d 0x18 0xffffffff 0xffffffff 0xffffffff 0xffffffff>;
};

And flash:

sudo ./flash -r -k bpmp-fw-dtb jetson-tx2 mmcblk0p1

…but

  • I get a SHA2 signature error on boot, and NX is stuck in boot loop
  • If I do a full install (flash everything) the bootloop disappears, but SSC is still enabled (measure by oscilloscope)

Can you provide the necessary changes to the bpmp-fw-dtb, as well as instructions to flash with signature?

Are you sure “sudo ./flash -r -k bpmp-fw-dtb jetson-tx2 mmcblk0p1” is the flash command you used? I mean since this is NX, shouldn’t you be using “jetson-xavier-nx-devkit” (for SD card based board) and “jetson-xavier-nx-devkit-emmc.” (for emmc based board) instead of ‘jetson-tx2’??

Anyway, NX has two slots
M.2 Key-E slot which is owned by C4 controller and
M.2 Key-M slot which is owned by C5 controller
Both controllers (C4 and C5) have different PLL sources. So, if the spread needs to be disabled for C4 then, please apply the following patch

diff --git a/include/parts/uphy/tegra194-bpmp-p2888-uphy.dtsi b/include/parts/uphy/tegra194-bpmp-p2888-uphy.dtsi
index 7f1a3fd..b0fa711 100644 (file)
 
--- a/include/parts/uphy/tegra194-bpmp-p2888-uphy.dtsi
+++ b/include/parts/uphy/tegra194-bpmp-p2888-uphy.dtsi
@@ -1,4 +1,4 @@
-
+#include <mach-t194/clk-t194.h>
 / {
        uphy {
                status = "okay";
@@ -6,4 +6,11 @@
                ufs-config = "UFS_x1_L1";
                nvhs-owner = "PCIE";
        };
+       clocks {
+                       clock@plle {
+                               clk-id = <TEGRA194_CLK_PLLE>;
+                               /* disable ssc on PLLE */
+                               pll_freq_table = <38400000 100000000 2 125 24 (-1) (-1) (-1) (-1)>;
+                       };
+       };
 };

If it is for the C5 controller, then, use the below patch

diff --git a/include/parts/uphy/tegra194-bpmp-p2888-uphy.dtsi b/include/parts/uphy/tegra194-bpmp-p2888-uphy.dtsi
index 7f1a3fd..b63d02e 100644 (file)
--- a/include/parts/uphy/tegra194-bpmp-p2888-uphy.dtsi
+++ b/include/parts/uphy/tegra194-bpmp-p2888-uphy.dtsi
@@ -1,4 +1,4 @@
-
+#include <mach-t194/clk-t194.h>
 / {
        uphy {
                status = "okay";
@@ -6,4 +6,11 @@
                ufs-config = "UFS_x1_L1";
                nvhs-owner = "PCIE";
        };
+       clocks {
+               clock@pllnvhs {
+                       clk-id = <TEGRA194_CLK_PLLNVHS>;
+                       /* disable ssc on PLLNVHS */
+                       pll_freq_table = <38400000 100000000 2 125 24 (-1) (-1) (-1) (-1)>;
+               };
+       };
 };

@vidyas,

Thanks for the reply!

I wrote jetson-tx2 in the flash command as a reference to the similar question for the X2. Sorry for the confusion. I am using the correct command in the lab.

I will try out these modifications tomorrow and report the results.

Again thanks!

Hi Vidyas,

We have modified the bpmp device tree to disable SSC. Thank you for this info. We are able to measure with oscilloscope that the spread is gone.

We have now done a simple test in SRNS mode - that is, root port (NX) talks with a generic endpoint without propagating the refclk. This seems to be successful and we are able to communicate with the endpoint.

Sorry to bother you with this, but is it possible for Nvidia to comment whether there is any risk involved in using the Tegra NX in SRNS mode? I assume this is not officially supported on your side.

Yes. you are correct that this is not officially supported.
Reasons being since the same PLL is used for both USB and PCIe and USB certification needs spread to be enabled, if you don’t care about USB much (i.e. the certification part) and ok to have any EMI issues that might come because of spread being disabled, then, it is ok to keep the spread disabled.

Again thanks for quick reply - we are OK with spread being disabled -
BUT the actual question is: does Nvidia officially support that the root port (NX) and endpoint work with independent 100 MHz reference clocks?

I.e.
[100 MHz refclk, no SSC] —> NX(RP) <— PCIe cabling without refclk —> ENDPOINT <— [100 MHz refclk, no SSC]

This is basically SRNS. Separate reference clocks without SSC.

For anyone that needs to do this on their NX kits (or similar):

Vidyas patches are from a git repo we do not have access to. So what you need to do:

  • Pay attention to the #include directive in the patch. While I cannot find <mach-t194/clk-t194.h>, I can find the clock id - TEGRA194_CLK_PLLNVHS by just grepping through everything after doing source_sync.sh. Use the clock ID from there and mind that you use the same clock ID that is for the L4T version you are using. E.g. source_sync.sh -d <YOUR_L4T_VERSION>, then “find . -type f -exec grep TEGRA194_CLK_PLLNVHS {} ;”

Then you device-tree-decompile the relevant bpmp in your Linux_for_Tegra folder (where flash.sh resides. For example:

  • If you do sudo ./flash.sh -r jetson-xavier-nx-devkit-emmc mmcblk0p1:

  • Then look at the jetson-xavier-nx-devkit-emmc.conf

  • cat jetson-xavier-nx-devkit-emmc.conf | grep source
    source ${LDK_DIR}/p3668.conf.common";

  • And if you look for bpmp in that file you will find the one you are flashing:

  • cat p3668.conf.common | grep -i bpmp
    BPFDTB_FILE=tegra194-a02-bpmp-p3668-a00.dtb;

  • Look for that file

  • find . -name tegra194-a02-bpmp-p3668-a00.dtb
    ./bootloader/t186ref/tegra194-a02-bpmp-p3668-a00.dtb
    ./bootloader/tegra194-a02-bpmp-p3668-a00.dtb

  • Then decompile one of these

  • dtc -o test.dts ./bootloader/t186ref/tegra194-a02-bpmp-p3668-a00.dtb

  • In test.dts, locate the “clock” parent and insert the patch, using the correct (!) clock id

    clocks {
    clock@pllnvhs {
    clk-id = <0xf3>;
    pll_freq_table = <0x249f000 0x5f5e100 0x02 0x7d 0x18 0xffffffff 0xffffffff 0xffffffff 0xffffffff>;
    };

  • Compile back to dtb and copy these back
    dtc -o test.dtb test.dts
    cp test.dtb ./bootloader/t186ref/tegra194-a02-bpmp-p3668-a00.dtb
    cp test.dtb ./bootloader/tegra194-a02-bpmp-p3668-a00.dtb

  • Noting I needed sudo in front of one of these.

  • Then flash the entire kit.
    sudo ./flash.sh -r jetson-xavier-nx-devkit-emmc mmcblk0p1

Note that we have verified the SSC off using oscilloscope.

Usually, it should work but to answer your question, Yes, Nvidia doesn’t support this configuration officially.
Please note that it is possible to get AER corrected errors if any of the ASPM states are enabled.

Thanks a lot for you feedback Vidyas :-)