Jetson TX1 PCIe X4 slot with extend cable or riser card

Autonomous Machines Jetson & Embedded Systems Jetson TX1
1

We have one issue with Jetson TX1 PCIe X4 slot as below.

When it’s used extend cable or riser card with PCIe Gen2 add-in card, Jetson TX1 can’t be boot-up into Ubuntu.
If it’s changed to PCIe Gen1 add-in card, it can be boot-up from our testing.

The requests form us and our end customer as list.

Can you help us to get SI report with Jetson TX1 PCIe X4 slot?
Do you have the test pattern commands for our SI measurement with PCIe X4 slot?
Do you have JTAG or debug tool with Jetson TX1 for confirmation?

Please help us to clarify this issue form FAE support.

2

In theory the Lauterbach Trace32 JTAG debugger works, but I can’t confirm yet (I have one here I’ve been trying to set up for a bit over a week…it probably will work with more effort, but it definitely isn’t “plug-n-play”). See this, look for Trace32:
[url]TRACE32®

3

We will get back to you on SI measurement info.
Meanwhile, can you please answer the following queries?
→ What type of Gen-2 PCIe add-on card are you connecting? what is its width? (I understand slot is x4, but is the width of the card also x4?)
→ Is it getting enumerated if connected without cable extension or raiser card?

4

Yes, it’s PCIe X4 Gen2 add-in card with Intel Ethernet server adapter I350-T2.
It can be confirmed as link.
http://ark.intel.com/zh-tw/products/59062/Intel-Ethernet-Server-Adapter-I350-T2
Yes, it’s fine without cable extension or riser card.

Please help us to check this problem.
Thanks a lot.

5

Do you have any link for the raiser card / extension cable also?
Wondering if the raiser card’s wire routing length is going beyond allowed range…!

6

The links are as below. The riser card is shorter than cable, but it still can’t be boot-up on Jetson TX1!
Please help us to confirm it.

Cable extension
https://world.tmall.com/item/39267207044.htm?spm=a1z3o.7695460.0.0.3RcKqQ&id=39267207044
Riser card
http://www.advantech.com.tw/products/1-2jkoyp/aimb-r4104-01a1e/mod_72954ebb-a3c7-48c0-aaa4-4af0568954af

7

Hey Egbert Lin,

what are you working ?

on boot press “i” that should continue the boot

8

Connecting extension cable seems to violate the PCIe spec defined routing length requirements.
With raiser card, Can you please connect a Gen-1 only device and see if that gets enumerated properly? This would give us some clue.

9

Additional info: Spread spectrum is enabled on the Jetson PCIe. A riser card isn’t really a remote PCIe node, but like a remote node, spread spectrum greatly reduces the available length extensions can reach. SS is really intended in this case to reduce RF noise created, and is a challenge for PCIe physical trace limitations.

10

What do those screen shots represent?Are these screenshots taken with/without riser card cable inserted?
=> You can check folder name taken with ubuntu version and the figures name taken with/without riser card inserted.

If kernel versions are same, whether it is 14.04 or 16.04 in the user space shouldn’t matter. Can we ask them for the output of ‘uname –a’ in both cases. Also, it would be good to know which packages (rel-23.2 / 24.2 Etc…) they flashed to get 14.04 and 16.04 respectively.
Does Intel LAN card come up on 14.04 if raiser card is not connected?
=> Ubuntu 14.04, rel-r24.1, but we re-built kernel to disable SSC function:
→ Linux tegra-ubuntu 3.10.96+ #3 SMP PREEMPT Wed Aug 31 15:25:33 CST 2016 aarch64 aarch64 aarch64 GNU/Linux
Ubuntu 16.04, rel-r24.2:
→ Linux tegra-ubuntu 3.10.96-tegra #1 SMP PREEMPT Thu Sep 8 13:48:54 PDT 2016 aarch64 aarch64 aarch64 GNU/Linux

Does this card come up in Gen-2 on 14.04 & 16.04 if raiser card is removed? If yes, I think, as our syseng folks predicted, it is something to do with routing length violations at higher speeds.
=> Yes, it can be boot-up with PEX8732 card when plug in PCIe X4 slot without riser card on Jetson TX1.

11

Hi,

Do you have any update for SI report with Jetson TX1 PCIe X4 slot?
We need it to double confirm the result.

12

PCIe signal interface requirements are already mentioned in “Jetson TX1 OEM Product Design Guide”
You can refer to
Table-20, 21 & 22

13

Hello,

We need PCIe sigtest report with PCIe X4 slot, not layout routing guide.
And we want to make sure disable SSC(spread spectrum clock) on TX1.
Please help us to confirm the code from our SW RD as below.

drivers/platform/tegra/tegra21_clocks.c

#define USE_PLLE_SS 0

#if USE_PLLE_SS
val = clk_readl(PLLE_SS_CTRL);
val &= ~(PLLE_SS_CNTL_CENTER | PLLE_SS_CNTL_INVERT);
val &= ~PLLE_SS_COEFFICIENTS_MASK;
val |= PLLE_SS_COEFFICIENTS_VAL;
clk_writel(val, PLLE_SS_CTRL);
val &= ~(PLLE_SS_CNTL_SSC_BYP | PLLE_SS_CNTL_BYPASS_SS);
pll_writel_delay(val, PLLE_SS_CTRL);
val &= ~PLLE_SS_CNTL_INTERP_RESET;
pll_writel_delay(val, PLLE_SS_CTRL);
#endif

include/linux/platform/tegra/clock.h

#ifdef CONFIG_ARCH_TEGRA_2x_SOC
#define USE_PLL_LOCK_BITS 0 /* Never use lock bits on Tegra2 /
#else
#define USE_PLL_LOCK_BITS 1 / Use lock bits for PLL stabiliation /
#define USE_PLLE_SS 0 / Use spread spectrum coefficients for PLLE /
#define PLL_PRE_LOCK_DELAY 2 / Delay 1st lock bit read after pll enabled /
#ifdef CONFIG_ARCH_TEGRA_3x_SOC
#define PLL_POST_LOCK_DELAY 50 / Safety delay after lock is detected /
#else
#define USE_PLLE_SWCTL 0 / Use s/w controls for PLLE /
#define PLL_POST_LOCK_DELAY 10 / Safety delay after lock is detected */
#endif
#endif

14

I’m yet to get PCIe sigtest report for x4 slot from our engineering team. I’ll update as soon as I receive it.
Regarding, disabling Spread Spectrum, your understanding in #13 is correct. Set USE_PLLE_SS to ‘0’ to disable SS for PLL-E (which is the source of PCIe REFCLK)

15

Hi,

Thanks for your reply.

Can you provide power sequence of Jetson TX1 by our customer’s request?
We needs this information to design power ON/OFF sequence on their system.
Please help us to get the datasheet ASAP.
Thanks.

16

Please refer to section-3.4 of JetsonTX1_OEM_Product_DesignGuide.pdf. It has covered all this information in detail.

17

Hi,

I’m facing the same question.
As you advised of #13 and #14,disabled the SSC, I tried, but i can’t get any more pcie info still, so can you received any info from your engineering team ?

18

>> I’m facing the same question.
Which question are you referring to here?

19

I tried use a wireless card with PCIe 1x on my Jetson Tx1, but when lspci -vvv,there is nothing show to me. and even i tried diabled the SSC, it has no affect.By the way, the wireless card is worked on my ubuntu normally .

My question is more clearly describe here:
Nothing showing up with lspci -v - Jetson TX1 - NVIDIA Developer Forums

I will apretiate for your step explanation， thanks very much.

20

There tried to disable PCIe SSC which only disable Kernel codes seems no influence.
Follow Spec 5.2.17 flag PLLE_SSCBYP found out uboot source code.
To mask clock.c “PLLE_SS_CNTL_SSCINC” relation codes that SSC seems be disabled (follow Source code changed:
to show change points).
But can’t be sured those way are correct and will no make any side event .
Hope the nVidia TX1 experts suggest us which best way for disable PCIe SSC.

Idea source:

Document: Tegra_X1_TRM_DP07225001_v1.2p.pdf
Register: 5.2.17 CLK_RST_CONTROLLER_PLLE_SS_CNTL_0
Flage: PLLE_SSCBYP: 0 enables spreading, 1 disables spreading


Source code changed:

#if 0 // mask PLLE_SSCBYP flag to disable PCIe Spread Spectrum Clock(SSC)
value = readl(NV_PA_CLK_RST_BASE + PLLE_SS_CNTL);
value &= ~PLLE_SS_CNTL_SSCINC(0xff);
value |= PLLE_SS_CNTL_SSCINC(1);
value &= ~PLLE_SS_CNTL_SSCINCINTR(0x3f);
value |= PLLE_SS_CNTL_SSCINCINTR(0x23);
value &= ~PLLE_SS_CNTL_SSCMAX(0x1fff);
value |= PLLE_SS_CNTL_SSCMAX(0x21);
value &= ~PLLE_SS_CNTL_SSCINVERT;
value &= ~PLLE_SS_CNTL_SSCCENTER;
value &= ~PLLE_SS_CNTL_BYPASS_SS;
value &= ~PLLE_SS_CNTL_SSCBYP;
writel(value, NV_PA_CLK_RST_BASE + PLLE_SS_CNTL);

/* 6. Wait 300 ns */

udelay(1);
value &= ~PLLE_SS_CNTL_INTERP_RESET;
writel(value, NV_PA_CLK_RST_BASE + PLLE_SS_CNTL);

#endif