TX2 using PCIx1 Instead of USB3 default (Config 1)

Hello,

We have created our own carrier card for the Tegra TX2. We would like to use the single lane PCI bus that is also pin muxed with the USB 3.0 by default. We were able to successfully change the pinmux using the excel spreadsheet, and generating the cfg file in python from the dtsi files, and finally flashing it using JetPack to the device.

This has enabled the power rails to our PCI device however we do not see the device listed at all in lspci -vv.

What I’m wondering is this…do we need to additionally configure dtb files to enable the PCI peripheral instead of the USB? Perhaps we only changed the pin muxxing but have not enabled the core functionality. Do we need to recompile the kernel to enable some feature?

Thanks in advance!

Please check documentation -> PDF Documents -> Platform Adaptation and Bring-Up Guide: Tegra Linux Driver Package for Jetson TX2 -> USB LANE MAPPING
https://developer.nvidia.com/embedded/dlc/l4t-documentation-28-1

Please be noted that ODMDATA has to be modified accordingly.

Thank you DaneLLL, I will give it a shot and report back.

Dane, I’m still beating my head on this one. Let me illustrate what I believe needs to be done and if you can correct me that would be great.

Overarching steps

  1. Adjust the pinmux to move UPHY USB_SS0_XXX to PEX1_XXX
  2. Adjust the pinmux to drive QSPI_IO2 to LOW to select mux as shown in OEM design guide shown below

  1. Adjust the pinmux to drive GPIO7 HIGH to select PEX1_CLK#REQ#

  1. Adjust the device tree file to include the additional PCIE lane and remove USB3??

  2. Set ODMDATA = 0x90000 to give lane ownership to PCIe only

  3. Flash image

Before I expand on what I’ve done for each step could you give me a quick yes looks good on steps 1-6?

Thanks

Hi GimpMaster, You want to run default carrier board in config #1? Or you are asking for HW design of your own carrier board?

Hello Dane, I first want to get it working on default carrier board in config #1. We build a simple little board that plugs into the mSATA port (J18) that converts it to a PCI 1X slot.

However our end goal is to use our own carrier board.

Hi GimpMaster,
To run config #1 on default board, please do

  1. ODMDATA = 0x90000
  2. Apply setting of xhci@3530000
phys = <&tegra_xusb_padctl TEGRA_PADCTL_PHY_UTMI_P(0)>;
		phy-names = "utmi-0";

to
tegra186-quill-p3310-1000-a00-00-base.dts
tegra186-quill-p3310-1000-c03-00-base.dts
tegra186-quill-p3310-1000-a00-plugin-manager.dtsi

  1. Disable usb2-std-A-port2 and usb3-std-A-port2
usb2-std-A-port2 {
				nvidia,lanes = "otg-1";
				nvidia,function = "xusb";
				nvidia,port-cap = <TEGRA_PADCTL_PORT_HOST_ONLY>;
				nvidia,oc-pin = <1>;
                                status = "disabled";
			};
			usb3-std-A-port2 {
				nvidia,lanes = "usb3-1";
				nvidia,port-cap = <TEGRA_PADCTL_PORT_HOST_ONLY>;
				nvidia,oc-pin = <1>;
                                status = "disabled";
			};

It should enable PEX1 and disable USB_SS0.

Thank you Dane, just some quick comments.

tegra186-quill-p3310-1000-a00-plugin-manager.dtsi contains 3 places with references to xhci that look like this…

under fragment-e3325-xusb .....

 override@1 {
                                target = <&{/xhci@3530000}>;
                                _overlay_ {
                                        phys = <&tegra_xusb_padctl TEGRA_PADCTL_PHY_UTMI_P(0)>,
                                                        <&tegra_xusb_padctl TEGRA_PADCTL_PHY_UTMI_P(1)>,
                                                        <&tegra_xusb_padctl TEGRA_PADCTL_PHY_USB3_P(1)>,
                                                        <&tegra_xusb_padctl TEGRA_PADCTL_PHY_UTMI_P(2)>,
                                                        <&tegra_xusb_padctl TEGRA_PADCTL_PHY_USB3_P(0)>;
                                        phy-names = "utmi-0", "utmi-1", "usb3-1", "utmi-2", "usb3-0";
                                };
                        };


under fragement-500-xusb-config

 override@0 {
                                target = <&{/xhci@3530000}>;
                                _overlay_ {
                                        phys = <&tegra_xusb_padctl TEGRA_PADCTL_PHY_UTMI_P(0)>,
                                                <&tegra_xusb_padctl TEGRA_PADCTL_PHY_UTMI_P(1)>,
                                                <&tegra_xusb_padctl TEGRA_PADCTL_PHY_UTMI_P(2)>,
                                                <&tegra_xusb_padctl TEGRA_PADCTL_PHY_USB3_P(0)>;
                                        phy-names = "utmi-0", "utmi-1", "utmi-2", "usb3-0";
                                };
                        };


finally under fragment-500-e3325-pcie

 override@0 {
                                target = <&{/xhci@3530000}>;
                                _overlay_ {
                                        phys = <&tegra_xusb_padctl TEGRA_PADCTL_PHY_UTMI_P(0)>,
                                                <&tegra_xusb_padctl TEGRA_PADCTL_PHY_UTMI_P(1)>;
                                        phy-names = "utmi-0", "utmi-1";
                                };
                        };

Do I want to modify all of these to what you have given?

Also for the usb2-std-A-port2 and usb3-std-A-port2 I see that only in this file…

quill/kernel-dts/tegra186-quill-p3310-1000-a00-00-base.dts

Let me know if that is correct.

Thank you very much for your support!!!

Dane,

I modified as best as I thought and changed all the overrides. I’m not seeing anything through lspci even with -vv. My dmesg looks like the following:

nvidia@tegra-ubuntu:/sys/firmware/devicetree/base$ dmesg | grep pci
[    0.142263] node /plugin-manager/fragment-500-pcie-config match with board >=3310-1000-500
[    0.142986] node /plugin-manager/fragment-500-e3325-pcie match with board >=3310-1000-500
[    0.143001] node /plugin-manager/fragment-500-e3325-pcie match with odm-data enable-pcie-on-uphy-lane0
[    0.261943] GPIO line 459 (pcie-lane2-mux) hogged as output/low
[    0.265388] iommu: Adding device 10003000.pcie-controller to group 50
[    6.516198] tegra-pcie 10003000.pcie-controller: 4x1, 1x1 configuration
[    6.525219] tegra-pcie 10003000.pcie-controller: PCIE: Enable power rails
[    6.533266] tegra-pcie 10003000.pcie-controller: probing port 0, using 4 lanes
[    6.546384] tegra-pcie 10003000.pcie-controller: probing port 2, using 1 lanes
[    7.081019] tegra-pcie 10003000.pcie-controller: link 0 down, retrying
[    7.499003] tegra-pcie 10003000.pcie-controller: link 0 down, retrying
[    7.979000] tegra-pcie 10003000.pcie-controller: link 0 down, retrying
[    7.987020] tegra-pcie 10003000.pcie-controller: link 0 down, ignoring
[    8.416917] tegra-pcie 10003000.pcie-controller: link 2 down, retrying
[    8.829027] tegra-pcie 10003000.pcie-controller: link 2 down, retrying
[    9.244191] tegra-pcie 10003000.pcie-controller: link 2 down, retrying
[    9.252973] tegra-pcie 10003000.pcie-controller: link 2 down, ignoring
[    9.259748] tegra-pcie 10003000.pcie-controller: PCIE: no end points detected
[    9.267422] tegra-pcie 10003000.pcie-controller: PCIE: Disable power rails

I’m not sure what it means (GPIO line 459 (pcie-lane2-mux) hogged as output/low) however I’m still not seeing anything. Any thoughts?

Hi GimpMaster, can you remove
fragment-500-pcie-config
fragment-500-e3325-pcie
fragment-500-e3325-pcie

in tegra186-quill-p3310-1000-a00-plugin-manager.dtsi? Seems like some settings are overwritten.

DaneLLL,

Is there any way to test that I have the correct settings once the system is booted? I saw in other posts you were talking about using devmem2?

Also do I need to configure the pinmux like I mentioned in my first post? Or does the dtb overwrite the pinmux that is loaded from MBL?

Thanks

Hi GimpMaster, configuring ODMDATA and device tree should be enough.
You can check the registers via devmem2
https://devtalk.nvidia.com/default/topic/1002494/jetson-tx2/usb-lane-mapping/post/5125933/#5125933

Also can check the device tree via xxd

nvidia@tegra-ubuntu:~$ xxd /proc/device-tree/xhci@3530000/phy-names
00000000: 7574 6d69 2d30 0075 746d 692d 3100 7574  utmi-0.utmi-1.ut
00000010: 6d69 2d32 0075 7362 332d 3000            mi-2.usb3-0.

Relative nodes:

/proc/device-tree/pcie-controller@10003000/
/proc/device-tree/pinctrl@3520000/pinmux/

Dane,

Please see the following…

nvidia@tegra-ubuntu:~$ xxd /proc/device-tree/xhci@3530000/phy-names
00000000: 7574 6d69 2d30 00                        utmi-0.

devmem2 shows the following:

Value at address 0x2520284 (0x7fad4e3284): 0x1
Value at address 0x2530284 (0x7f9d4e2284): 0x1
Value at address 0x2540284 (0x7fb587c284): 0x1
Value at address 0x2550284 (0x7fab782284): 0x1
Value at address 0x2560284 (0x7f78fce284): 0x1
Value at address 0x2570284 (0x7f7c847284): 0x2

I’m not sure what you want me to do with these proc folders.

/proc/device-tree/pcie-controller@10003000/
/proc/device-tree/pinctrl@3520000/pinmux/

In the /proc/device-tree/pinctrl@3520000/pinmux/usb3-std-A-port2

status using xdd is:
00000000: 6469 7361 626c 6564 00 disabled.

however under /proc/device-tree/pcie-controller@10003000/pci@2,0
00000000: 6469 7361 626c 6564 00 disabled.

the other pci@1,0 and pci@3,0 are OK it says

Thoughts?

Hi GimpMaster, your setting looks correct. It is still not working?

You can apply the same to your carrier board. If your board doesn’t have ‘usb2-micro-AB’, you need to remove it also.

Thank you Dane. I will try with my carrier card. Perhaps our MSATA->PCIE converter board is not correct.

Can you tell me a little more details on what those different register settings mean just for my knowledge? Thanks

repeat

These are the UPHY registers to check their mapping for difference IPs

Absolute address = 0x02520284 NV_ADDRESS_MAP_UPHY_LANE0_BASE
Absolute address = 0x02530284 NV_ADDRESS_MAP_UPHY_LANE1_BASE
Absolute address = 0x02540284 NV_ADDRESS_MAP_UPHY_LANE2_BASE
Absolute address = 0x02550284 NV_ADDRESS_MAP_UPHY_LANE3_BASE
Absolute address = 0x02560284 NV_ADDRESS_MAP_UPHY_LANE4_BASE
Absolute address = 0x02570284 NV_ADDRESS_MAP_UPHY_LANE5_BASE

Bits (2 ~ 0) specify the IP owning the lane
SEL:
0 = XUSB
1 = PCIE
2 = SATA
3 = MPHY

Copied from https://devtalk.nvidia.com/default/topic/1002494/jetson-tx2/usb-lane-mapping/post/5125933/#5125933

Thank you,

Do you know why it would say this?

/proc/device-tree/pcie-controller@10003000/pci@2,0
00000000: 6469 7361 626c 6564 00 disabled.

?

It is PCIe#1_0, which is disabled in config#1.

Hello DaneLLL,

Just giving you some follow up. I decided to stop trying to get it working on the dev kit through the m.2 interface and jumped right to our custom carrier card and I see my PCI devices!!!

I’m seeing other issues now but I’ll create a new post. Thank you for your help.