Regarding PCIE clock of Jetson TX2

I wish to comfirm the following two things:

  1. Do REFCLK± pins conform to HCSL electrical standards ? (i can’t see any useful info regarding this, neither DS nor OEM Design Guide). Otherwise please point out to what electrical standard shall REFCLK± conform?

  2. Please show how REFCLK± pins are terminated on the TX2 module (if any). Or are these two pins
    simply not terminated at all ?

By the way, i have noticed that the PCIE reference clock is not always present, is there a way to turn on the clock
as soon as the module gets powered on even when no endpoint is attached

I’ll get back to you with the REFCLK questions soon.
Regarding, keeping REFCLK alive even when there is no PCIe endpoint device connected, please use the following patch.

diff --git a/drivers/pci/host/pci-tegra.c b/drivers/pci/host/pci-tegra.c
index 63c0e343d388..52c46b68981e 100644
--- a/drivers/pci/host/pci-tegra.c
+++ b/drivers/pci/host/pci-tegra.c
@@ -2194,7 +2194,7 @@ static bool tegra_pcie_port_check_link(struct tegra_pcie_port *port)
                tegra_pcie_port_reset(port);
        } while (--retries);
  
-       return false;
+       return true;
 }
  
 static void tegra_pcie_apply_sw_war(struct tegra_pcie_port *port,

The default behavior of TX2 is that, if there is no PCIe endpoint detected, it powers down the PCIe controller (thereby shutting off the REFCLK as well). The above patch changes that behavior.

Our engineering team says that irrespective of the source (i.e. LVPECL, LVDS, CML,HCSL) REFCLK should always conform to AC and DC specifications as mentioned by the spec.
FWIW, TX2 does conform to the spec and the source in case of TX2 is CML.

1 Like

thanks for your explanation!

But you still didn’t show whether REFCLK± are already terminated on Jetson TX2 module …
Please clarify this …
Thanks ~

I see one item in the DS that you posted
i.e., Zc-dc = 40Ω ~ 60Ω
can I interpret this as each pin of REFCLK± is terminated with a resistor of ~50Ω
as shown in the figure bellow


please help me with this, thank you

Yes. You can proceed with that understanding.

Hi,
I have designed an electronic card that has Nvidia TX2 and a Intel FPGA Stratix V. Their PCIe design is referencing the schematics of Jetson TX2 development kit and Intel FPGA Stratix V development kit which do not have any termination resistors in their respective schematics for PCIe reference clock. In an earlier chat Jetson TX1/TX2 PCIE differential reference clock type , a question was posted about the I/O standard of TX2 PCIe reference clock and it was replied by Nvidia moderator Trumany that it is HCSL. Within Intel FPGA, I configured it to receiver PCIe reference clock as HCSL I/O standard.

When I encountered intermittent behaviour of the PCIe interface : there are times when TX2 cannot detect the Intel FPGA PCIe device with lspci command, there are also times when during TX2 boot up, there are PCIe error messages running non-stop. I did a measurement of the TX2 PCIe clock with an oscilloscope and discovered that the TX2 PCIe clock is not HCSL. A HCSL clock should be toggling between 0mV and 700mV. The measured has an positive offset voltage of about 600mV and toggling of about 150mV swing, riding on top of the 600mV. This signal appears to be more like a LVDS signal although LVDS should have a 1.2V positive offset voltage. As LVDS receiver can tolerate quite a wide variation of input offset voltage. I reconfigured Stratix V FPGA to receive the clock as LVDS receiver. With this change, the intermittent situations improve significantly with many previously failed electronic cards working. However, there are still failed cases.

I also measured the PCIe clock resistance to ground with a multimeter and received a very high resistance value. This means there is no 50 ohm termination to ground within TX2 for this PCIe differntial clock as shown in Phoenixlee’s last post.

According to Intel FPGA support, its PCIe Hard IP should be configured to receive PCIe clock as HCSL receiver and not LVDS receiver. May I know if it is possible to configure TX2 to output a HCSL PCIe clock ? Or can Nvidia advise how to convert its PCIe clock signal to HCSL ? Or can I provide a separate HCSL clock source to Intel FPGA, not using TX2 PCIe clock source ? Please provide details of the TX2 PCIe clock specification to clarify.

regards,
Hui Peng

HCSL signal range is typically between 0 and 800 mV and Tegra PEX_CLK typical output is within this range.

There is common mode termination resistor that can be enabled/disabled so the input termination on the receiver side is not necessary. Tegra default is to enable the common mode resistor with a typical value of 50 Ohm. Common mode resistor may need to be disabled depend on if the receiver has internal termination or not.

Do you have termination enabled on the FPGA device if it is supported? If not, you may need to include it externally.

Ideally, it shouldn’t matter what kind of source (LVPECL / LVDS / CML / HCSL) is used to generate the REFCLK as long as the REFCLK itself conforms to the PCIe spec and in Tegra’s case, it does.

Hi Trumany,

Please provide guide on how to enable the 50 ohm common mode termination resistors within Nvidia TX2.

regards,
Hui Peng

Please refer to the tuning guide: http://developer.nvidia.com/embedded/dlc/tx2-series-interface-tuning-compliance-testing-guide

T_PCIE2_PADS_REFCLK_BIAS

Hi Trumany,

With reference to the attached crops from the guide, do you have more explanation than what is provided in the guide ?

Specifically, how to set these 2 registers : T_PCIE2_PADS_REFCLK_BIAS & T_PCIE2_PADS_REFCLK_CFG0 ?

For my design, the REFCLK is driven from Jetson TX2 to Intel FPGA clock input pin. On the PCB, it is just a differential pair traces with 100 ohm differential impedance. Intel FPGA is configured to receive HCSL with 100 ohm termination. In my PCB design, there are no 2x 50 ohm termination to ground as required by HCSL source to load. There are also no 2x 33 ohm serial resistors as shown in following circuit from a HCSL clock source :
HCSL oscillator circuit

Please advise the values to apply to these 2 registers. Please also provide the method to set these 2 registers through the Ubuntu Linux environment.

regards,
Hui Peng

Hi,

The devmem tool from busybox should be able to write the register value.

Hi WayneWWW,

Thanks for the information.

May I know is there a document that has the complete information of these registers ?
My TX2 is configured to Configuration 5 which has 3 PCIe buses. In the guide, register T_PCIE2_PADS_REFCLK_CFG0 can only configured for 2 REFCLKs, how about the 3rd REFCLK register ? I saw in TX2 Technical Reference Manual, there is a header file ardev_t_pcie2_pads.h that probably defines all these registers. May I know how to get it ?

regards,
Hui Peng

Hi,
I found in Parker_TRM_DP07821001p.pdf page 3624 in the crop above that there are actually 2 registers PCIE2_PADS_REFCLK_CFG0_0 and PCIE2_PADS_REFCLK_CFG0_1 for the 3 PCIe reference clocks of the 3 PCIe buses.
Please help to find out the register address of these 2 registers.

regards,
Hui Peng

Hi,
Using Busybox devmem command, I managed to read back and write the registers. I tried the following :

  1. read back of register T_PCIE2_PADS_REFCLK_BIAS (address 0x100030D0) with 0x28, this is the recommended setting from the Tuning Guide
  2. read back of register T_PCIE2_PADS_REFCLK_CFG0_0(address 0x100030C8) with 0x80B880B8
  3. read back of register T_PCIE2_PADS_REFCLK_CFG0_1(address 0x100030CC) with 0x000480B8

I have a oscilloscope to measure 1 of the 3 PCIe reference clock on my Printed Circuit Board, its measured signal is a 600mV positive offset with a 150mV switching clock riding on the positive offset.

  1. when register T_PCIE2_PADS_REFCLK_CFG0_1 (address 0x100030CC) was written with value 0x0004F0B8, the current driver is supposed to increase current drive from 10.4mA to 16mA. This current is supposed to flow through the 50 ohm termination resistors to create voltage increase from 520mV to 800mV. But in observation, the scope only showed an increase in voltage from around 150mV to 200mV. Can Nvidia explain why there is only marginal increase in voltage ?

  2. I saw in TX2 Technical Reference Manual, there is a header file ardev_t_pcie2_pads.h that probably defines all the PCIe PADS registers. May I know how to get it ?

  3. Please also advise how to set the change so that a newly compiled kernel can have changes to these registers. It is because if the PCIe reference clock is not stable to the FPGA, there will be a lot of PCIe error messages during booting and it will deny TX2 from successfully boot up. It will not be possible to use busybox devmem command if TX2 cannot even boot up.

regards,
Hui Peng

Hi Hui Peng,

Please help to open a new topic for your issue, Thanks

Hi kayccc,

I have opened a new topic, link as follows :

regards,
Hui Peng