How to set config#4 with TX1 & devboard

Hello

I have 2 questions.

We are creating custom boards. And I’d like to establish a procedure to change to Config # 4 using the developer board first.

Looking at Tegra_Linux_Driver_Package_TX1_Adaptation_Guide.pdf, I made dtb by reflecting the modifications in “R28 Example 2” on tegra210-jetson-cv-base-p2597-2180-a00.dts.However PCIe Configuration is “4x1, 1x1 configuration”(dmesg log #23).

The first question is how to make it “1x1, 1x1”.
In the reference forum, there were 4_1 and 2_1 possible combinations of PCIe controllers.
Is there any other place to change to make Config #4(“1x1, 1x1”)?

The second question was, in TX 2, there was a means of using devmem 2 to check if Config changed.
In TX 1, which address should we refer to in devmem 2?

reference forum
https://devtalk.nvidia.com/default/topic/1025927
https://devtalk.nvidia.com/default/topic/1022604/jetson-tx1/r28-1-how-to-change-to-pcie-config-4-

### ~//hardware/nvidia/platform/t210/jetson/kernel-dts/jetson-platforms/tegra210-jetson-cv-base-p2597-2180-a00.dts
pinctrl@7009f000 {
	status = "okay";
	pinctrl-0 = <&tegra_padctl_uphy_pinmux_default>;
	pinctrl-names = "default";
	tegra_padctl_uphy_pinmux_default: pinmux {
		usb2-micro-AB-xusb {
			nvidia,lanes = "otg-0";
			nvidia,function = "xusb";
			nvidia,port-cap = <TEGRA_PADCTL_PORT_OTG_CAP>;
			nvidia,usb3-port-fake = <2>;
		};
		usb2-std-A-port0 {
			 nvidia,lanes = "otg-2";
			 nvidia,function = "xusb";
			 nvidia,port-cap = <TEGRA_PADCTL_PORT_HOST_ONLY>;
		};
		usb3-std-A-port0 {
			 nvidia,lanes = "uphy-lane-5";
			 nvidia,function = "usb3";
			 nvidia,usb3-port = <1>;
			 nvidia,usb2-map = <2>;
			 nvidia,port-cap = <TEGRA_PADCTL_PORT_HOST_ONLY>;
		};
		usb2-eth {
			 nvidia,lanes = "otg-1";
			 nvidia,function = "xusb";
			 nvidia,port-cap = <TEGRA_PADCTL_PORT_HOST_ONLY>;
		};
		usb3-eth {
			 nvidia,lanes = "uphy-lane-6";
			 nvidia,function = "usb3";
			 nvidia,usb3-port = <0>;
			 nvidia,usb2-map = <1>;
			 nvidia,port-cap = <TEGRA_PADCTL_PORT_HOST_ONLY>;
		};
		hsic {
			 nvidia,lanes = "hsic-0";
			 nvidia,function = "hsic";
		};
		pcie {
			 nvidia,lanes = "uphy-lane-4";
			 nvidia,function = "pcie";
			 nvidia,pcie-controller = <0>;
			 nvidia,pcie-lane-select = <TEGRA_PADCTL_PCIE_LANE_X4>;
		};
		pcie-m2 {
			nvidia,lanes = "uphy-lane-0";
			nvidia,function = "pcie";
			nvidia,pcie-controller = <1>;
			nvidia,pcie-lane-select = <TEGRA_PADCTL_PCIE_LANE_X1>;
			status = "disabled";
		};
		sata {
			 nvidia,lanes = "uphy-lane-7";
			 nvidia,function = "sata";
		};

		usb2-standalone {
			nvidia,lanes = "otg-3";
			nvidia,function = "xusb";
			nvidia,port-cap = <TEGRA_PADCTL_PORT_HOST_ONLY>;
		};
		usb3-standalone {
			nvidia,lanes = "uphy-lane-3";
			nvidia,function = "usb3";
			nvidia,usb3-port = <2>;
			nvidia,usb2-map = <3>;
			nvidia,port-cap = <TEGRA_PADCTL_PORT_HOST_ONLY>;
		};
	};
};

xusb@70090000 {
	status = "okay";
	extcon-cables = <&vbus_gpio_extcon 1>;
	extcon-cable-names = "id";
	#extcon-cells = <1>;

	phys =	<&tegra_padctl_uphy TEGRA_PADCTL_UPHY_UTMI_P(2)>,
		<&tegra_padctl_uphy TEGRA_PADCTL_UPHY_USB3_P(1)>,
		<&tegra_padctl_uphy TEGRA_PADCTL_UPHY_UTMI_P(1)>,
		<&tegra_padctl_uphy TEGRA_PADCTL_UPHY_USB3_P(0)>,
		<&tegra_padctl_uphy TEGRA_PADCTL_UPHY_UTMI_P(0)>,
		<&tegra_padctl_uphy TEGRA_PADCTL_UPHY_UTMI_P(3)>,
		<&tegra_padctl_uphy TEGRA_PADCTL_UPHY_USB3_P(2)>;
	phy-names = "utmi-2", "usb3-1", "utmi-1", "usb3-0", "utmi-0", "utmi-3", "usb3-2";

	nvidia,pmc-wakeup =
		<&tegra_pmc PMC_WAKE_TYPE_EVENT 41 PMC_TRIGGER_TYPE_HIGH>,
		<&tegra_pmc PMC_WAKE_TYPE_EVENT 44 PMC_TRIGGER_TYPE_HIGH>;
};
### dmesg log
[    0.480521] io scheduler deadline registered
[    0.480848] io scheduler cfq registered
[    0.486357] tegra21x-padctl-uphy 7009f000.pinctrl: padctl mmio start 0x000000007009f000 end 0x000000007009ffff
[    0.486403] tegra21x-padctl-uphy 7009f000.pinctrl: TEGRA_FUSE_SKU_CALIB_0 0x81e9c11
[    0.486437] tegra21x-padctl-uphy 7009f000.pinctrl: TEGRA_FUSE_USB_CALIB_EXT_0 0x4
[    0.488001] tegra21x-padctl-uphy 7009f000.pinctrl: UTMI-0 is used by XUSB
[    0.488038] tegra21x-padctl-uphy 7009f000.pinctrl: using UTMI port 0 for otg
[    0.488068] tegra21x-padctl-uphy 7009f000.pinctrl: UTMI-2 is used by XUSB
[    0.488105] tegra21x-padctl-uphy 7009f000.pinctrl: uphy_lane = 5, set usb3_lanes = 0x20
[    0.488148] tegra21x-padctl-uphy 7009f000.pinctrl: USB3 port 1 maps to USB2 port 2
[    0.488182] tegra21x-padctl-uphy 7009f000.pinctrl: UTMI-1 is used by XUSB
[    0.488215] tegra21x-padctl-uphy 7009f000.pinctrl: uphy_lane = 6, set usb3_lanes = 0x60
[    0.488255] tegra21x-padctl-uphy 7009f000.pinctrl: USB3 port 0 maps to USB2 port 1
[    0.488298] tegra21x-padctl-uphy 7009f000.pinctrl: uphy_lane = 4, set pcie_lanes = 0x10
[    0.488346] tegra21x-padctl-uphy 7009f000.pinctrl: uphy_lane = 7, set sata_lanes = 0x80
[    0.488380] tegra21x-padctl-uphy 7009f000.pinctrl: UTMI-3 is used by XUSB
[    0.488412] tegra21x-padctl-uphy 7009f000.pinctrl: uphy_lane = 3, set usb3_lanes = 0x68
[    0.488452] tegra21x-padctl-uphy 7009f000.pinctrl: USB3 port 2 maps to USB2 port 3
[    0.488887] tegra21x-padctl-uphy 7009f000.pinctrl: mailbox is not ready yet
[    0.489937] tegra-pwm 7000a000.pwm: PWM clk can sleep in ops
[    0.491049] tegra-dfll-pwm 70110000.pwm: DFLL pwm-rate: 12800000
[    0.491852] Adding domain pcie-pd to PM domain mc-clk-pd
[    0.496673] tegra-pcie 1003000.pcie-controller: 4x1, 1x1 configuration
[    0.496743] tegra-pcie 1003000.pcie-controller: failed to get PHY: -517
[    0.496770] tegra-pcie 1003000.pcie-controller: failed to get PHYs: -517
[    0.500323] tsec 54500000.tsec: initialized
[    0.501628] tsec 54100000.tsecb: initialized
[    0.504620] nvdec 54480000.nvdec: initialized
[    0.510085] falcon 54340000.vic: initialized

We have below guidance in adaptation:

• nvidia,lanes 
In this case, USB 3.0 port 2 is using Lane 3, so set nvidia,lanes = "uphy-lane-3" under usb3-standalone sub-node and set nvidia,lanes = "otg-3" of usb2- standalone sub node.  
Since PCIe configuration changes from 1x1+1x4 to 1x1+1x1, change nvidia,lanes of sub-node pcie from 4 lanes("uphy-lane-1", "uphy-lane-2", "uphy-lane-3", "uphy-lane4") to "uphy-lane-4".
pcie {
	 <b>nvidia,lanes = "uphy-lane-4";</b>
	 nvidia,function = "pcie";
	 nvidia,pcie-controller = <0>;
	 nvidia,pcie-lane-select = <TEGRA_PADCTL_PCIE_LANE_X4>;
};

You should have configured it correctly.

For TX1, you don’t need to configure ODMDATA, so you can check the correctness of device tree via xxd:
https://devtalk.nvidia.com/default/topic/1023160/jetson-tx2/tx2-using-pcix1-instead-of-usb3-default-config-1-/post/5208073/#5208073

Thank you for your reply.

I corrected it correctly as documented.

The latest version of Tegra_Linux_Driver_Package_TX1_Adaptation_Guide.pdf is v1.5, is not it?

$ xxd /proc/device-tree/pinctrl@7009f000/pinmux/pcie/nvidia\,lanes 
00000000: 7570 6879 2d6c 616e 652d 3400            uphy-lane-4.

$ xxd /proc/device-tree/pinctrl@7009f000/pinmux/pcie-m2/status 
00000000: 6469 7361 626c 6564 00                   disabled.

$ xxd /proc/device-tree/pinctrl@7009f000/pinmux/usb2-standalone/nvidia\,lanes 
00000000: 6f74 672d 3300                           otg-3.

$ xxd /proc/device-tree/pinctrl@7009f000/pinmux/usb2-standalone/nvidia\,function 
00000000: 7875 7362 00                             xusb.

$ xxd /proc/device-tree/pinctrl@7009f000/pinmux/usb2-standalone/nvidia\,port-cap 
00000000: 0000 0001                                ....

$ xxd /proc/device-tree/pinctrl@7009f000/pinmux/usb3-standalone/nvidia\,lanes 
00000000: 7570 6879 2d6c 616e 652d 3300            uphy-lane-3.

xxd /proc/device-tree/pinctrl@7009f000/pinmux/usb3-standalone/nvidia\,function 
00000000: 7573 6233 00                             usb3.

$ xxd /proc/device-tree/pinctrl@7009f000/pinmux/usb3-standalone/nvidia\,usb3-port 
00000000: 0000 0002                                ....

$ xxd /proc/device-tree/pinctrl@7009f000/pinmux/usb3-standalone/nvidia\,usb2-map 
00000000: 0000 0003                                ....

$ xxd /proc/device-tree/pinctrl@7009f000/pinmux/usb3-standalone/nvidia\,port-cap 
00000000: 0000 0001                                ....

$ xxd /proc/device-tree/xusb@70090000/phys
00000000: 0000 0060 0000 0012 0000 0060 0000 0001  ...`.......`....
00000010: 0000 0060 0000 0011 0000 0060 0000 0000  ...`.......`....
00000020: 0000 0060 0000 0010 0000 0060 0000 0013  ...`.......`....
00000030: 0000 0060 0000 0002                      ...`....

$ xxd /proc/device-tree/xusb@70090000/phy-names
00000000: 7574 6d69 2d32 0075 7362 332d 3100 7574  utmi-2.usb3-1.ut
00000010: 6d69 2d31 0075 7362 332d 3000 7574 6d69  mi-1.usb3-0.utmi
00000020: 2d30 0075 746d 692d 3300 7573 6233 2d32  -0.utmi-3.usb3-2
00000030: 00

Hi chironeko0527,
The change looks good, but default developer board is not in config #4 and cannot be used for verification.

Hi DaneLLL

I know that config # 4 can not be verified on the developer board.
So, it is a correct move to become “4x1, 1x1” within dmesg?

[    0.496673] tegra-pcie 1003000.pcie-controller: 4x1, 1x1 configuration
[    0.496743] tegra-pcie 1003000.pcie-controller: failed to get PHY: -517
[    0.496770] tegra-pcie 1003000.pcie-controller: failed to get PHYs: -517

[    4.469506] tegra-pcie 1003000.pcie-controller: probing port 0, using 4 lanes
[    4.471622] tegra-pcie 1003000.pcie-controller: probing port 1, using 1 lanes

If we create a custom board conforming to config# 4, it should be displayed as “1x1, 1x1”?

Yes, it is correct. you always have to keep

nvidia,pcie-lane-select = <TEGRA_PADCTL_PCIE_LANE_X4>;

And change nvidia,lanes accordingly.

nvidia,lanes = "uphy-lane-4";

Hi DaneLLL.

Sorry, I want to get an answer about following question.

No, it is always 4x1, 1x1.

Thank you for your reply.

I have question about USB.
Config#4 has USB3.0 x2.

In device tree, usb3-port1 is “usb3-std-A-port0”(Lane5), and usb-port2 is “usb3-standalone”(Lane3)
what is how to check that Lane 3 has been changed to USB_SS# 2? Only look device tree?

I tried lsusb. I found usb3.0-port1.

nvidia@tegra-ubuntu:~$ lsusb -t
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci-tegra/4p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Vendor Specific Class, Driver=r8152, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci-tegra/5p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
        |__ Port 1: Dev 3, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
        |__ Port 3: Dev 4, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
        |__ Port 3: Dev 4, If 1, Class=Human Interface Device, Driver=usbhid, 1.5M

Hi chironeko0527,
Yes, only look at device tree.

Hi DaneLLL,

I’m sorry to hear the same thing.

If TX1 is set to config#4 on default developer board, USB_SS#2 is not displayed as usb-port2 within lsusb.
Is it correct?

Not sure but I think lsusb only shows physical HW ports.