Jetson TX2 custom carrier PCIe lane4 not working

stfl · June 4, 2018, 4:18pm

We have custom Carrier Board for TX2.
We use Configuration #4 from the Platform adaption guide.

PCIe#1_0 and all USB Ports are working
PCIe#0_0 where we connect a Rennesas USB Host controller for additional USB3.0 does not work.

tegra186-quill-p3310-1000-c03-00-base.dts

pcie-controller@10003000 {
		status = "okay";
    // PCIe#0_0
		pci@1,0 {
			nvidia,num-lanes = <1>;
			status = "okay";
		};
    // PCIe#1_0
		pci@2,0 {
			nvidia,num-lanes = <1>;
			status = "okay";
		};
		pci@3,0 {
			nvidia,num-lanes = <1>;
			status = "disabled";
		};
	};

xudc@3550000 {
		status = "okay";
		phys = <&tegra_xusb_padctl TEGRA_PADCTL_PHY_UTMI_P(0)>,
      <&tegra_xusb_padctl TEGRA_PADCTL_PHY_UTMI_P(1)>,
      <&tegra_xusb_padctl TEGRA_PADCTL_PHY_UTMI_P(2)>,
      <&tegra_xusb_padctl TEGRA_PADCTL_PHY_USB3_P(0)>,
      <&tegra_xusb_padctl TEGRA_PADCTL_PHY_USB3_P(1)>,
      <&tegra_xusb_padctl TEGRA_PADCTL_PHY_USB3_P(2)>;
		phy-names = "usb2", "usb2", "usb2", "usb3", "usb3", "usb3";
	};

xhci@3530000 {
		phys = <&tegra_xusb_padctl TEGRA_PADCTL_PHY_UTMI_P(0)>,
			<&tegra_xusb_padctl TEGRA_PADCTL_PHY_UTMI_P(1)>,
			<&tegra_xusb_padctl TEGRA_PADCTL_PHY_UTMI_P(2)>,
      <&tegra_xusb_padctl TEGRA_PADCTL_PHY_USB3_P(0)>,
      <&tegra_xusb_padctl TEGRA_PADCTL_PHY_USB3_P(1)>,
      <&tegra_xusb_padctl TEGRA_PADCTL_PHY_USB3_P(2)>;
		phy-names = "utmi-0", "utmi-1", "utmi-2", "usb3-0", "usb3-1", "usb3-2";
	};

	pinctrl@3520000 {
		pinmux {
      usb2-0-debug {
				nvidia,lanes = "otg-0";
				nvidia,function = "xusb";
				nvidia,port-cap = <TEGRA_PADCTL_PORT_OTG_CAP>;
				nvidia,oc-pin = <0>;
        status = "okay";
			};

			usb2-2-front {
				nvidia,lanes = "otg-1";
				nvidia,function = "xusb";
				nvidia,port-cap = <TEGRA_PADCTL_PORT_HOST_ONLY>;
        nvidia,oc-pin = <1>;
        status = "okay";
			}; 

			usb2-5-aux {
        nvidia,lanes = "otg-2";
        nvidia,function = "xusb";
        nvidia,port-cap = <TEGRA_PADCTL_PORT_HOST_ONLY>;
        status = "okay";
			}; 

			usb3-2-front {
        nvidia,lanes = "usb3-0";
        nvidia,port-cap = <TEGRA_PADCTL_PORT_HOST_ONLY>;
        nvidia,oc-pin = <1>;
        status = "okay";
      };

      usb3-1-front {
        nvidia,lanes = "usb3-1";
        nvidia,port-cap = <TEGRA_PADCTL_PORT_HOST_ONLY>;
        /* nvidia,oc-pin = <?>; */ // USB#1.OC#     | B22 | GPIO3_PB.05  | 1  | TEGRA_MAIN_GPIO(1, 5)
        status = "okay";
      };

      usb3-5-aux {
        nvidia,lanes = "usb3-2";
        nvidia,port-cap = <TEGRA_PADCTL_PORT_HOST_ONLY>;
        status = "okay";
    }; 
  };

I updated power tree in tegra186-quill-power-tree-p3310-1000-a00-00.dtsi

pinctrl@3520000 {
		vbus-0-supply = <&vdd_usb0_5v>;
		vbus-1-supply = <&vdd_usb1_5v>;
		/* vbus-2-supply = <&vdd_usb2_5v>; */
		vbus-2-supply = <&battery_reg>;
		vbus-3-supply = <&battery_reg>;
		vddio-hsic-supply = <&battery_reg>;
		avdd_usb-supply = <&spmic_sd3>;
		vclamp_usb-supply = <&spmic_sd2>;
		avdd_pll_erefeut-supply = <&spmic_sd2>;
	};

I disabled the some fragments in the plugin-manager.

/* fragment-500-pcie-config {
/* fragment-500-xusb-config {

The device tree in /proc/device-tree looks like it’s finally taking my settings and not overwriting anything anymore.

With dev2mem I get following values. The PCI/USB Muxing seams to be configured correctly.

Value at address 0x2520284 (0x7f9a4e9284): 0x0  > USB_SS0  OK
Value at address 0x2530284 (0x7f78a87284): 0x0  > USB_SS1  OK
Value at address 0x2540284 (0x7f839d5284): 0x0  > USB_SS2  OK
Value at address 0x2550284 (0x7fb22c6284): 0x1  > PCIe1_0  OK
Value at address 0x2560284 (0x7f7813b284): 0x1  > PCIe0_0  OK
Value at address 0x2570284 (0x7f9a3e9284): 0x2  > SATA     OK

I updated the pmic cfg for c00, c01, c02, c03 and c04 according to https://devtalk.nvidia.com/default/topic/1001443/jetson-tx2/extlinux-conf-fdt-no-longer-used-for-dtb-file-specification-/post/5130849/#5130849

I set ODM=0x7090000 at flash and I rewrote the board config:

process_board_version()
{
	local board_id="${1}";
	local board_version="${2}";
	local bdv=${board_version^^};
	local bid=${board_id^^};
	local uboot_build=000;
	local fromfab="-a00";
	local tofab="-c03";		# default = C03
	local pmicfab="-c00";		# default = C00
	local bpfdtbfab="-c00";		# default = C00
	local tbcdtbfab="-c03";		# default = C03
	local kerndtbfab="-c03";	# default = C03

	#
	# Board version is Alpah-Numeric value.
	#
	echo "Board ID(${board_id}) version(${board_version}) " >/dev/stderr;
	if ! [[ "${bdv}" =~ ^[A-Z0-9][0-9][0-9]$ ]]; then
		bdv="${DEFAULT_FAB:-Z99}";
	fi;

	ODMDATA=0x7090000;
  pmicfab="-c04";
  bpfdtbfab="-c04";

	sub_var_token "BPFDTB_FILE"    "${fromfab}" "${bpfdtbfab}";
	sub_var_token "PINMUX_CONFIG"  "${fromfab}" "${tofab}";
	sub_var_token "PMIC_CONFIG"    "${fromfab}" "${pmicfab}";
	sub_var_token "PMC_CONFIG"     "${fromfab}" "${tofab}";
	sub_var_token "PROD_CONFIG"    "${fromfab}" "${tofab}";
	sub_var_token "BOOTROM_CONFIG" "${fromfab}" "${tofab}";
	sub_var_token "TBCDTB_FILE"    "${fromfab}" "${tbcdtbfab}";
	sub_var_token "DTB_FILE"       "${fromfab}" "${kerndtbfab}";

	if [ "${USE_UBOOT}" -eq 1 ]; then
    DFLT_KERNEL_IMAGE="bootloader/t186ref/myboard/";         # refering to my u-boot build
		DFLT_KERNEL_IMAGE+="${uboot_build}/u-boot.bin";
	fi;
}

...
...

BCT="--sdram_config";
BINSARGS="--bins \"";
DEV_PARAMS="emmc.cfg";
SCR_CONFIG="minimal_scr.cfg";
SCR_COLD_BOOT_CONFIG="mobile_scr.cfg";
MISC_CONFIG="tegra186-mb1-bct-misc-si-l4t.cfg";
# PINMUX_CONFIG="tegra186-mb1-bct-pinmux-quill-p3310-1000-a00.cfg";
PINMUX_CONFIG="tegra186-mb1-bct-pinmux-myboard.cfg";                          # my pinmux config generated from pinmux.xlsm
PMIC_CONFIG="tegra186-mb1-bct-pmic-quill-p3310-1000-a00.cfg";
# PMC_CONFIG="tegra186-mb1-bct-pad-quill-p3310-1000-a00.cfg";
PMC_CONFIG="tegra186-mb1-bct-pad-myboard.cfg";                                   
PROD_CONFIG="tegra186-mb1-bct-prod-quill-p3310-1000-a00.cfg";
BOOTROM_CONFIG="tegra186-mb1-bct-bootrom-quill-p3310-1000-a00.cfg";

BPFDTB_FILE=tegra186-a02-bpmp-quill-p3310-1000-a00-00-te770d-ucm2.dtb;

In dmesg I see that the pcie is configuring and probing port 0 and 1 while only port 1 goes active.

nvidia@tegra-ubuntu:~$ dmesg | grep -i pci
[    6.106695] tegra-pcie 10003000.pcie-controller: PCIE: Enable power rails
[    6.117566] tegra-pcie 10003000.pcie-controller: probing port 0, using 1 lanes
[    6.134706] tegra-pcie 10003000.pcie-controller: probing port 1, using 1 lanes
[    6.558783] tegra-pcie 10003000.pcie-controller: link 0 down, retrying
[    7.084215] tegra-pcie 10003000.pcie-controller: link 0 down, retrying
[    7.524768] tegra-pcie 10003000.pcie-controller: link 0 down, retrying
[    7.534483] tegra-pcie 10003000.pcie-controller: link 0 down, ignoring
[    7.542222] tegra-pcie 10003000.pcie-controller: PCI host bridge to bus 0000:00
[    7.551361] pci_bus 0000:00: root bus resource [mem 0x50100000-0x57ffffff]
[    7.559179] pci_bus 0000:00: root bus resource [mem 0x58000000-0x7fffffff pref]
[    7.568111] pci_bus 0000:00: root bus resource [bus 00-ff]
[    7.574484] pci_bus 0000:00: root bus resource [io  0x1000-0xffff]
[    7.581571] pci 0000:00:02.0: [10de:10e6] type 01 class 0x060400
[    7.588651] pci 0000:00:02.0: PME# supported from D0 D1 D2 D3hot D3cold
[    7.596670] pci 0000:00:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    7.596838] pci 0000:01:00.0: [1912:0015] type 00 class 0x0c0330
[    7.596916] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00001fff 64bit]
[    7.597154] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[    7.597461] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[    7.597495] pci 0000:00:02.0: BAR 8: assigned [mem 0x50100000-0x501fffff]
[    7.597500] pci 0000:01:00.0: BAR 0: assigned [mem 0x50100000-0x50101fff 64bit]
[    7.597521] pci 0000:00:02.0: PCI bridge to [bus 01]
[    7.597528] pci 0000:00:02.0:   bridge window [mem 0x50100000-0x501fffff]
[    7.597596] pcieport 0000:00:02.0: enabling device (0000 -> 0002)
[    7.597731] pcieport 0000:00:02.0: Signaling PME through PCIe PME interrupt
[    7.597733] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt
[    7.597738] pcie_pme 0000:00:02.0:pcie01: service driver pcie_pme loaded
[    7.597812] aer 0000:00:02.0:pcie02: service driver aer loaded
[    7.597872] pci 0000:01:00.0: enabling device (0000 -> 0002)
[    7.607405] tegra-pcie 10003000.pcie-controller: speed change : Gen-1 -> Gen-2

boot2.log (102 KB)

WayneWWW · June 6, 2018, 6:14am

According to your log, I can see there is i2c error. Is this from port 0?

[    6.196767] tegra-i2c 3190000.i2c: rx dma timeout txlen:28 rxlen:128
[    6.205374] tegra-i2c 3190000.i2c: --- register dump for debugging ----
[    6.213159] tegra-i2c 3190000.i2c: I2C_CNFG - 0x22c00
[    6.218791] tegra-i2c 3190000.i2c: I2C_PACKET_TRANSFER_STATUS - 0x10001
[    6.220727] dhd_module_init in

Also, how is the result of

/proc/device-tree/pcie-controller@10003000/pci@1,0/status
and
/proc/device-tree/pcie-controller@10003000/pci@2,0/status

stfl · June 6, 2018, 8:55am

We got it to work by setting pci@1 needed num-lanes=<2>.
I looks like Tegra Lane4 is actually using PCIe#0_1 ?!

pcie-controller@10003000 {
		status = "okay";
		pci@1,0 {
			nvidia,num-lanes = <2>;
			nvidia,disable-clock-request;
			status = "okay";
		};
		pci@2,0 {
			nvidia,num-lanes = <1>;
			status = "okay";
		};
		pci@3,0 {
			nvidia,num-lanes = <1>;
			status = "disabled";
		};
	};

WayneWWW · June 6, 2018, 10:09am

stfl,

I think that is related to your hardware design. Are all of your usb devices still working under such 2,1,1 configuration?

stfl · June 6, 2018, 11:41am

Yes. Thank you for pointing this out. We just noticed that USB_SS1 is not working anymore.

So, for the hardware design we connect the PCIe0 to PEX0 H44,H45 A44,A45 and PCIe1 to PEX2 F40,F41 C40,C41.
According to the OEM Guide that should be the correct layout to use PCIe0_0 (lane4)?!

Best regards, Stefan

stfl · June 11, 2018, 8:56am

Hi. Ok So we figured out that USB_SS1 was just not working on that one specific carrier board caused by a manufacturing issue. All USB ports work now on the carrier but we’re still on the 2,1,1 setup.

Can you confirm that the routing PEX0 H44,H45 A44,A45 is correct? Why aren’t we getting signals there in a 1,1,1 setting?

WayneWWW · June 12, 2018, 2:55am

Hi stfl,

The OEM DG indicates the correct routing is H44, H45, E44, E45. Could you check it again?

stfl · June 13, 2018, 11:09am

Ok, just checked again. I mixed it up. A44 and A45 is the CLK and we’re using E44, E45 as TX+TX-
The attached schematic shows our configuration.

(The polarity inversion on PEX2 works btw)

WayneWWW · June 14, 2018, 3:00am

This user has confirmed the sw config for config 4 is working.

https://devtalk.nvidia.com/default/topic/1030635/jetson-tx2/tx2-config-4-for-usb-lane-mapping/post/5242906/#5242906

Please note that he needed usb3.0 otg, but you don’t. Thus, you don’t need to change those "
xudc@3550000" and "xotg " and “usbcd”

Topic		Replies	Views
In the L4T R28 for TX1 the USB lane mapping is bug? Jetson TX1	15	1657	October 18, 2021
TX2 USB Lane Mapping configs #4 Jetson TX2	33	5918	June 6, 2018
TX2 using PCIx1 Instead of USB3 default (Config 1) Jetson TX2	26	6102	October 1, 2019
TX2 USB lane map config #1 Fail Jetson TX2	19	813	October 18, 2021
PCIe bridge issues with R24.1 Jetson TX1	7	1010	March 21, 2018
TX2 USB/PCIe Lane Mapping configs #3(JetPack 3.3) Jetson TX2	9	734	October 18, 2021
USB lane mapping config 1 on TX2 Dev Kit board Jetson TX2	20	1518	October 18, 2021
PCIe doesn't work on my custom board with a TX2 4GB Jetson TX2 pcie , board-design	18	1441	October 18, 2021
JETSON TX2 PCIe Jetson TX2	4	1819	October 18, 2021
TX2 Custom carrier board cannot recognize all PCIe devices Jetson TX2 pcie	20	969	October 18, 2021

Jetson TX2 custom carrier PCIe lane4 not working

Related topics