Hello everyone, I’m working on a PCIe setup using two NVIDIA Jetson Orin devkits (JetPack r35.6.0). One device is configured as a PCIe Endpoint (EP) and the other as a Root Complex (RC). So far, I’ve successfully established communication using the pci_epf_nv_test module — I was able to perform BAR0 read/write transactions from the RC to the EP. Now, I’m trying to implement DMA transfers from the EP to memory on the RC using the pci-epf-dma-test driver. I followed NVIDIA’s official documentation here: 🔗 NVIDIA Developer Guide – PCIe Endpoint Mode Current Setup: EP side is configured and running pci-epf-dma-test. RC side is rebooted and successfully enumerates the EP device. $ lspci 0005:00:00.0 PCI bridge: NVIDIA Corporation Device 229a (rev a1) 0005:01:00.0 RAM memory: NVIDIA Corporation Device 229a Problem: When I try to run the test from the EP side using: cat /sys/kernel/debug/141a0000.pcie_ep_epf_dma_test/edmalib_test I get: cat: ...edmalib_test: Operation not permitted And the corresponding dmesg log includes: [ 452.102846] pcie_dma_epf tegra_pcie_dma_epf.0: RP DMA address is null EP’s dmesg before that confirm that the driver runs: [ 296.518691] tegra194-pcie 141a0000.pcie_ep: pcie_dma_epf_write_msi_msg: MSI idx: 0 data: 608 [ 296.518736] tegra194-pcie 141a0000.pcie_ep: pcie_dma_epf_write_msi_msg: MSI idx: 1 data: 609 [ 296.518750] tegra194-pcie 141a0000.pcie_ep: pcie_dma_epf_write_msi_msg: MSI idx: 2 data: 610 [ 296.518765] tegra194-pcie 141a0000.pcie_ep: pcie_dma_epf_write_msi_msg: MSI idx: 3 data: 611 [ 296.518778] tegra194-pcie 141a0000.pcie_ep: pcie_dma_epf_write_msi_msg: MSI idx: 4 data: 612 [ 296.518791] tegra194-pcie 141a0000.pcie_ep: pcie_dma_epf_write_msi_msg: MSI idx: 5 data: 613 [ 323.946344] pcie_dma_epf tegra_pcie_dma_epf.0: BAR0 phy_addr: f0000000 size: 10000000 [ 350.702783] pcie_dma_epf tegra_pcie_dma_epf.0: BAR0 phy_addr: f0000000 size: 10000000 It seems that the EP does not have a valid DMA target address from the RC. I assume the RC must somehow expose or communicate its physical memory region for the EP to perform DMA into — but I can’t figure out how. I initially thought I needed to load pci-epf-dma-test on the RC side as well, but I couldn’t link it to any controller, and I now suspect it may not be meant for use on the RC. My Questions: Does the RC need to run a driver or perform additional configuration to provide a DMA-accessible memory region to the EP? Is there an NVIDIA-provided Root Port companion driver for pci-epf-dma-test? What is the correct way for the EP to discover or be given the RC-side DMA address? I’d really appreciate any clarification or guidance. Thanks!

Hi everyone, Just wanted to share a quick update on my progress with PCIe communication between two Jetson Orin devkits (JetPack r35.6.0), where one board acts as the PCIe Endpoint and the other as the Root Complex. I found a driver in JetPack r36.3.0 which includes a working DMA example in pcie_dma_epf. I managed to get a stable DMA communication channel working! What’s Working Now: Using the updated driver from version 36, I followed the pcie_dma_epf example closely. Achieved reliable connection and successful bi-directional DMA between the EP and RC. My current custom EPF DMA driver implementation is achieving ~22 Gbps sustained throughput in the test setup. Still a “naive” version — no advanced buffer management or transfer pipelining yet. I’m working on further optimizing this performance.

Hello @gilad.ben.or, Any updates?

Connection to Orin AGX Devkits - pcie

Robotics & Edge Computing Jetson Systems Jetson AGX Orin

kayccc June 18, 2025, 5:39am 4

Please refer to AGX Orin Devkit PCIE bandwidth test - Jetson & Embedded Systems / Jetson AGX Orin - NVIDIA Developer Forums to see if able to help. Thanks

Topic		Replies	Views
PCIe ep Test Fail on AGX orin:RP DMA address is null .Version:R36.3 Jetson AGX Orin pcie	14	575	November 21, 2024
Enabling Orin Dev Kit PCIe EP mode Jetson AGX Orin pcie	22	6545	December 20, 2022
PCIe Communication Between 2 Jetson Agx Orin Jetson AGX Orin pcie	19	717	March 27, 2025
PCIe endpoint dma driver usage example Jetson AGX Orin pcie	3	726	June 11, 2024
Fail to Testing Bidirectional Data Transfer Jetson AGX Orin pcie	9	1257	November 3, 2022
Jetson AGX Orin Dev Kit PCIe (Orin RP to Orin EP) Shared RAM Memory Not Working Jetson AGX Orin pcie	6	1182	May 10, 2023
PCIE test probelem Jetson Orin NX pcie	9	1185	January 22, 2025
How to use pci-epf-dma-test between two Nvidia AGX Xavier Jetson AGX Xavier pcie , performance	9	1184	May 16, 2024
DMA to PCIe BARs on Jetson Orin Jetson AGX Orin pcie	4	1671	July 31, 2023
Custom Driver for Data transmission Over PCIe Jetson AGX Orin pcie	1	154	May 7, 2025

Connection to Orin AGX Devkits - pcie

Related topics