PCIe C4 not working

[rhao]

Hello,

I am trying to use PCIe C4 lane that are connected to UPHY 8 and 9 on my custom board.
My dts file is like that:

pcie@14160000 {
	compatible = "nvidia,tegra194-pcie", "snps,dw-pcie";
	power-domains = <0x3 0x12>;
	reg = <0x0 0x14160000 0x0 0x20000 0x0 0x36000000 0x0 0x40000 0x0 0x36040000 0x0 0x40000>;
	reg-names = "appl", "config", "atu_dma";
	status = "okay";
	#address-cells = <0x3>;
	#size-cells = <0x2>;
	device_type = "pci";
	num-lanes = <0x4>;
	linux,pci-domain = <0x4>;
       .............

i get the following error dmesg:
[ 9.143387] tegra-pcie-dw 14160000.pcie: Setting init speed to max speed
[ 9.143395] tegra-pcie-dw 14160000.pcie: unable to find phy entries
[ 9.143542] tegra-pcie-dw 14160000.pcie: DT parsing failed: -22
[ 9.143681] tegra-pcie-dw: probe of 14160000.pcie failed with error -22
[ 9.143928] tegra-pcie-dw 141a0000.pcie: Setting init speed to max speed
[ 9.249833] OF: PCI: host bridge /pcie@141a0000 ranges:
[ 9.249851] OF: PCI: IO 0x3a100000…0x3a1fffff → 0x3a100000

---

I modified this file: tegra194-p2888-0000-a00.dtsi
as explained in that post:

How to configure UPHY8 to PCIe x1

Hi, vidyas. The interface of the C4 controller has been designed on the custom carrier board, the Tx/Rx ,PERST#,CLKREQ# & REFCLK is fine. How to enable the PCIe C4 controller node in the software? Now, only uphy7 can recognize i210, and uphy8 can’t recognize i210 network card. Thanks.

---

I tried to change the ODMDATA value to different values that I found in forum posts:
ODMDATA=0x4B191000;
or ODMDATA=0x9290000;
or ODMDATA=0x9191000

With no effect so far I still get this error.

Does anyone has a hint on how to make PCIe C4 work at all?

---

And one very basic question… I really don’t understand how to make the changes in the dtsi file being effective…
What do I have to do after changing tegra194-p2888-0000-a00.dtsi???
I rerun a kernel compile with:

make ARCH=arm64 O=$TEGRA_KERNEL_OUT tegra_defconfig
make ARCH=arm64 O=$TEGRA_KERNEL_OUT -j4
cp $TEGRA_KERNEL_OUT/arch/arm64/boot/Image kernel/
cp -r $TEGRA_KERNEL_OUT/arch/arm64/boot/dts kernel/

Is that needed to use the modified dtsi??? Or do I need to do something else? I really can’t fine documentation on that. Can someone point to the right document?

Many thanks,
raphael

[vidyas]

What is the release you are using? Is it Kernel-4.9 or Kernel-5.10?
If you are using kernel-4.9, please add the following

                phys = <&p2u_8>,
                      <&p2u_9>;

                phy-names = "pcie-p2u-0", "pcie-p2u-1";

If you are using Kernel-5.10, add the following instead

                phys = <&p2u_hsio_8>,
                      <&p2u_hsio_9>;

                phy-names = "p2u-0", "p2u-1";

Please also share the log and describe the new issue if it doesn’t work after making the above changes.

[rhao]

Hello,

I use kernel 4.9

uname -a
Linux tegra-ubuntu 4.9.253-tegra #1 SMP PREEMPT Mon Jul 4 15:34:39 CEST 2022 aarch64 aarch64 aarch64 GNU/Linux

I managed to make the changed to the device tree. It now shows the PCI C4.
I am just not sure if the procedure I use is correct… it seems to be like hacking until it works.

This is the way I procede:

1. modify sources/hardware/nvidia/platform/t19x/galen/kernel-dts/common/tegra194-p2888-0000-a00.dtsi
2. recompile kernel

make ARCH=arm64 O=$TEGRA_KERNEL_OUT tegra_defconfig
make ARCH=arm64 O=$TEGRA_KERNEL_OUT -j4

3. copy the kernel and device tree

cp $TEGRA_KERNEL_OUT/arch/arm64/boot/Image kernel/
cp -r $TEGRA_KERNEL_OUT/arch/arm64/boot/dts kernel/
cp $TEGRA_KERNEL_OUT/arch/arm64/boot/dts/* kernel/dtb

(the last copy is the one that I had missing before…)

4. I have more changes to the device tree that I need to do (for USB otg → host)
   For that I modify the dtb like that:

dtc -I dtb -O dts -o temp.dts tegra194-p2888-0001-p2822-0000.dtb
vi temp.dts
dtc -I dts -O dtb -o tegra194-p2888-0001-p2822-0000.dtb temp.dts

So this works to see the pci interface (lspci works) but it doesn’t seems correct as a production method.

Can anyone point me to a different method ???

---

Then my problem is that I cannot compile/install my PCIe driver.
[ 5815.931976] xillybus_core: disagrees about version of symbol module_layout
[ 5833.404819] xillybus_core: disagrees about version of symbol module_layout

I think I need to do an apt update/upgrade.
But the apt update/upgrade destroys my device tree changes. I tried upgrading with

apt-mark hold nvidia-l4t-kernel-dtbs

But not more success to compile/install the driver.

---

Is there a good documentation on the right procedure?
Am I missing something?

Many thanks
raph