How to program and use PCIe x16 slot in Xavier

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Xavier
1

Hi,
I want to communicate between two xavier board using pcie interface i.e through x16 PCIe slot. So I configured one as Root complex and other as endpoint in its device tree. Also I have written necessary drivers for it. I have enabled endpoint through config interface but it did not detected in root complex.

I found one major thing in both boards i.e PCIe controller 5 is not listed when i give lspci but in device tree i made changes in 0x141a000 node in device tree which corresponds to pcie controller 5.
I have attached my dtsi files for both xavier
root_complex.txt (557.4 KB)
endpoint.txt (557.4 KB)

3

Which side are you talking about here? The RP side or EP?

4

Please check my both RP and EP device tree files. In the Root complex side EP is not detected

5

How did you enable your PCIe EP on your EP side? Modifying a device tree is not sufficient.

I expect more steps here. Also, actually if you modify ODMDATA, then it will got enabled by default as our software will handle that. You don’t even need to modify device tree by yourself.

Another question here is which Jetpack release are you using?

6

I set ODMDATA value 0x9191000 for endpoint and 0x9190000 for root complex. I’m using L4T 32.5.1

7

Then did you start to run the steps on EP mentioned by document? The driver needs to be up first so that RP could detect it in lspci.

8

Yes I followed same steps on EP mentioned by document and then i turned on RP. when i gave lspci, my endpoint was not detected

9

I followed the configfs steps on EP and then I turned on RP

10

@WayneWWW
Here is lspci of my root complex:

rootcomplex
rootcomplex1024×184 70.1 KB

lspci of endpoint:

endpoint
endpoint1024×184 83.3 KB

If you see lspci, in both boards controller 5 i.e 0x141a0000 is not listed

11

Hi,

It is normal that it won’t show up on EP.

Rel-32.5.1 is a quite old release that I am not even sure EP is supported on that version.

Is it possible to move to some latest branches like rel-35.6?

12

Hi @WayneWWW ,
I tried with rel 35.6.0 and in that also same result. EP is not detected in RP. Whereas when i connect different PCIe EP device it gets listed but Xavier as EP is not detected

13

Ok. Could you share me the result of cat /proc/interrupts on rel-35.6 EP when you try to enable the EP?

I am pretty sure rel-35.6 AGX Xavier shall have this supported. Need to do some checks here.

14

Hi @WayneWWW ,
Please find the cat /proc/interrupts log. I have taken three times the log. One is after turning on EP , second is after enabling EP and third is after turning on RP.
before_enable.txt (12.9 KB)
after_enable.txt (12.9 KB)
after_turning_on_RP.txt (12.9 KB)

15

@WayneWWW ,
I got below LTSSM error, after i turned on RP.

ltssm_error
ltssm_error1024×317 133 KB

16

please try to share your full log as text file but not such screenshot first.

17

dmesg_log_ep.txt (84.7 KB)
I have uploaded dmesg log of EP in txt file.

18

Hi @WayneWWW ,
Did you find any information on /proc/interrupts log and dmesg logs?

19

Hi,

The last time we saw such “LTSSM state: 0x2018 timeout: -110” is due to hardware problem on the cable as this post mentioned.