I am working on a custom carrier board with Jetson AGX Thor (P3834-0008) and a Xilinx FPGA connected via PCIe C3. I have enabled PCIe C3 by modifying the device tree (uphy0-config=6, status=okay) and the XDMA driver (v2025.2.0) loads successfully. The PCIe device is detected by lspci, and the XDMA driver binds to it (Kernel driver in use: xdma). However, any DMA transfer (read or write) to the FPGA results in a Connection timed out error, even though the interrupt count is incrementing.
Hardware and software environment
JetPack version: R38.4.0 (L4T R38.4.0)
Kernel version: output of uname - r
Development board: self-developed carrier board, unofficial DevKit (Custom Carrier Board, based on P4071)
FPGA：Xilinx FPGA with XDMA IP
XDMA driver version: Xilinx XDMA v2025.2.0
PCIe configuration: C3 enabled, UPHY0-config changed from 0x07 to 0x06, UPHYVNet cleared
Completed troubleshooting steps
PCIe device correctly recognized (lspci - d 10ee: display device)
Control channel (/dev/xdma0_control) reads and writes normally (reg_rw can read 0x1fc00006)
Interrupts have been allocated (lspci - v shows IRQ 435), and when attempting DMA transfer, the interrupt count in/doc/interrupts will increase from 48 to 54 (an increase of 6 times)
Attempted to use 64 bit addresses (0x1000000 and 0x1100000000) for DMA transfer, both timed out
Read XDMA IP core status registers (0x2044, 0x2048, 0x204C), all return values are 0
Key log: PCIe bus error in kernel log after DMA timeout:
The PCIe link has been established, the driver has been bound, and the interrupt has been triggered, but why does the DMA request still timeout?
Does the PCIe Bus Error: NonFatalErr and xdma: engine_ded_dump: status=0x0000000 in the log indicate that the problem is on the FPGA side (such as AXI bus response, address decoding, or XDMA IP core configuration)?
When enabling PCIe C3 on a self-developed carrier board, besides modifying uphy0 config and status, are there any other necessary hardware (such as Pinmux, reset/clock signals) or software configurations (such as IOMMU/SMMU) that need to be adjusted? For example, I saw in the document that it is necessary to check settings such as PERST #, REFCLK, and hot debug programmable.
A community developer mentioned that on Orin, PCIe DMA needs to handle SMMU/IOMMU correctly, otherwise it will trigger EMEM address decode error. Do similar issues exist on Thor?