Use tegra-pcie-ep-mem.c(on root complex) to test the endpoint device failed in dma_read()

I’m trying to use PCIe to communication between two xaviers. One is in the default mode (root complex). Another is configured as endpoint mode.

After doing all the stuffs in post:
https://devtalk.nvidia.com/default/topic/1048611/jetson-agx-xavier/use-pcie-to-communicate-between-two-xaviers-ep-reports-quot-uphy-init-failed-for-pcie-ep-22-quot-/post/5321765/?offset=6#5322270

The rc-xavier can recognize the ep-xavier:
- Screenshot for command "lspci"on rc-xavier:

- Screenshot for command "dmesg"on ep-xavier:

And according to https://devtalk.nvidia.com/default/topic/1039469/jetson-agx-xavier/how-another-cpu-communicate-with-xavier-through-pcie-solved-/post/5301264/#5301264, I was trying to use tegra-pcie-ep-mem.c to test the endpoint device.

Then I run the command below on rc-xavier:

sudo cat /sys/kernel/debug/tegra_pcie_ep/write

And an error returns: “Input/output error
And the dmesg information is:

Do I need to write a kernel driver for “endpoint test function” in ep-xavier side? Or anything else I need to do in ep-xavier side?

Hi zhuce_cgf,

PCIe configuration steps on EP device:

cd /sys/kernel/config/pci_ep/
mkdir functions/pci_epf_nv_test/func1
echo 0x104c > functions/pci_epf_nv_test/func1/vendorid
echo 0xb500 > functions/pci_epf_nv_test/func1/deviceid
echo 16 > functions/pci_epf_nv_test/func1/msi_interrupts
ln -s functions/pci_epf_nv_test/func1 controllers/141a0000.pcie_ep/
echo 1 > controllers/141a0000.pcie_ep/start

Hi carolyuu,

Do I need to update package to “JetPack 4.2 — L4T R32.1” to run the command you gave me?
Because I can’t find pci_epf_nv_test related source code in L4T R32.1.

Hi zhuce_cgf,

Yes, you need use JetPack-4.2 and change ODM data to 0x9191000.

sudo vim p2972-0000.conf.common
change ODM data to 0x9191000
Flash image

OK, Thanks. And do I need to replace “jetson xavier” with “p2972-0000-devkit-pcie-ep” when flashing the image:

flash.sh p2972-0000-devkit-pcie-ep mmcblk0p1

Hi zhuce_cgf,

Yes, you need flash the image with jetson-xavier.conf

sudo vim p2972-0000.conf.common
change ODM data to 0x9191000
sudo ./flash jetson-xavier mmcblk0p1

OK, Thanks. But why the document “Tegra_Linux_Driver_Package_AGX_Xavier_Adaptation_Guide_v1-1.pdf” said that:

p2972-0000-devkit-pcie-ep.conf is simply a copy of the standard jetson-xavier.conf with the ODMDATA value already changed to enable endpoint mode. Editing jetson-xavier.conf (actually, the file it includes; p2972-0000.conf.common) to change the ODMDATA does the same thing as flashing with p2972-0000-devkit-pcie-ep.conf. The current L4T release unfortunately doesn’t include p2972-0000-devkit-pcie-ep.conf.

I change the system of ep-xavier to “Jetpack 4.2” and do whole process again. New issue is found. I have created a new post to it:
https://devtalk.nvidia.com/default/topic/1049041/jetson-agx-xavier/how-to-test-xavier-which-is-in-endpoint-mode/post/5323955/#5323955
And in this new post, I have tested the endpoint successfully. Actualy “tegra-pcie-ep-mem.c” is not used anymore in Jetpack 4.2.