Pcie@14180000 correct register offsets

[elitewompa]

Brief System Description

• JetPack 5.1.4 L4T 35.6.0
• NVidia Jetson AGX Orin Module 64GB - Non Industrial Version

We are trying to bring up PCIe root port C0
We noticed that the device tree contains the following values for controller dependent register offsets
from tegra234-soc-pcie.dtsi

from live device tree
nvidia,cfg-link-cap-l1sub = <0x1b0>;
nvidia,cap-pl16g-status = <0x174>;
nvidia,cap-pl16g-cap-off = <0x188>;
nvidia,event-cntr-ctrl = <0x1c4>;
nvidia,event-cntr-data = <0x1c8>;
nvidia,dl-feature-cap = <0x2f8>;
nvidia,ptm-cap-off = <0x304>;

However these offsets do not match what is documented in linux-tegra/inidia/Documentation/devicetree/bindings/pci/nvidia,tegra19x-pcie.txt

Which values should be used, the ones we see in the documentation or the ones we see in tegra234-soc-pcie.dtsi?

[WayneWWW]

Does this really affect anything for your bring up?

[elitewompa]

One would think that offsets to registers would be important to have correct otherwise the driver would read/write to incorrect registers.

Can you tell us which set of values are the correct?

The larger problem follows:
We are using the C0 root port on Orin.
This is connected via 1x lane width to a Marvell Switch.

based on lspci -vv output we observe that our link is up.
See Capabilities - LnkSta

LnkSta: Speed 8GT/s (ok), Width x1 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-

However when we try to read a register via the pcie bus we only get back 0xFFFFFFFF

We checked the PCIE_RP_APPL_DEBUG_0 register for C0 and observed the following

It appears that the LTSSM is in the S_DETECT_QUIET state. This does not make sense based on what lspci -vv showed above (it showed that the link was up)

So we started to look around to see what might be wrong and ran into these conflicting register values. Who knows maybe that might be the problem?

We welcome your debug suggestions.

[WayneWWW]

Could you share your full dmesg and full lspci -vvv result as text attachment?

[WayneWWW]

@elitewompa

PCIE_RP_APPL_DEBUG_0

For this, you should dump 0x141800d0.

[elitewompa]

dmesg output
dmesg.txt (69.9 KB)

lspci -vvv output
lspci-vvv.txt (8.7 KB)

[elitewompa]

dumped 0x141800d0

This is the LTSSM state S_DISABLED_ENTRY
From what i understand SW disabled the link?
hmmmmmm

may have something to do with this line from dmesg
[ 19.973939] pci 0000:01:00.0: AER: can’t recover (no error_detected callback)

[WayneWWW]

Hi,

bit 3:8 from the result is 010001 so the hex value here is 0x11. It should be S_L0 but not S_DISABLED_ENTRY. The LTSSM value is in hex. Not decimal.

[elitewompa]

@WayneWWW

S_L0

Good catch.
State S_L0 is where we want to be … so this is a piece of good news.

The problem still persists with read / write registers via the PCI link. No matter what register we read we only receive 0xFFFFFFFF

[elitewompa]

Going back to our original question

The device tree contains the following values for controller dependent register offsets
from tegra234-soc-pcie.dtsi

from live device tree
nvidia,cfg-link-cap-l1sub = <0x1b0>;
nvidia,cap-pl16g-status = <0x174>;
nvidia,cap-pl16g-cap-off = <0x188>;
nvidia,event-cntr-ctrl = <0x1c4>;
nvidia,event-cntr-data = <0x1c8>;
nvidia,dl-feature-cap = <0x2f8>;
nvidia,ptm-cap-off = <0x304>;

However these offsets do not match what is documented in linux-tegra/inidia/Documentation/devicetree/bindings/pci/nvidia,tegra19x-pcie.txt

Which values should be used, the ones we see in the documentation or the ones we see in tegra234-soc-pcie.dtsi?

[WayneWWW]

I am still checking this internally.

But I think that doesn’t matter in this issue at all.
You are not the first one bringing up C0 here. None of previous user ever need to configure this to enable PCIe devices.

[elitewompa]

Ok unless you say otherwise we will leave the values as they are.

FYI Since we do not know the source of the problem, We are starting a conversation with Marvell to help debug this register read/write issue.