Hello, any support on this topic, especially for the DMA speed test?
I followed this topic, but can’t figure out how to see the channel, size, etc. fields under /sys/kernel/debug/pcie@141a0000/ on the EP side. They don’t appear.
To attempt to solve this, I added these fields to tegra_defconfig:
CONFIG_PCIE_TEGRA=y
CONFIG_PCIE_RP_DMA_TEST=y
CONFIG_PCIE_TEGRA_DW=y
CONFIG_PCIE_TEGRA_HOST=y
CONFIG_PCIE_TEGRA_DW_DMA_TEST=y
Then I modified the source by defining CONFIG_PCIE_TEGRA_DW_DMA_TEST in these files:
nvidia/drivers/pci/host/pcie-tegra-dw.c
nvidia/drivers/pci/dwc/pcie-tegra.c
Then, I re-compiled kernel & re-flashed both my Orin AGX EP and Xavier NX RP.
I then boot both EP and RP. lspci enumeration, memory-mapped data transfer with busybox , and virtual Ethernet-over-PCIe still work properly.
On EP, under /sys/kernel/debug/pcie@141a0000/, I now see the following:
apply_flr
apply_sbr
aspm_state_cnt
ep_rid
perf_test
target_speed
apply_pme_turnoff
apply_speed_change
dma_size
flr_rid
sanity_test
However: When I attempt the DMA speedtest with cat perf_test, I get the following error in dmesg
[ 326.096457] tegra194-pcie 14100000.pcie: edma_submit_direct_tx: DD WR CH: 0 TO
[ 326.103966] tegra194-pcie 14100000.pcie: perf_test: DD WR, SZ: 267386880 B CH: 0 failed
[ 326.496852] irq 65: nobody cared (try booting with the "irqpoll" option)
[ 326.503794] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G OE 5.10.120-tegra #1
[ 326.503799] Hardware name: Unknown Jetson AGX Orin Developer Kit/Jetson AGX Orin Developer Kit, BIOS 4.1-33958178 08/01/2023
[ 326.503806] Call trace:
[ 326.503837] dump_backtrace+0x0/0x1d0
[ 326.503845] show_stack+0x30/0x40
[ 326.503870] dump_stack+0xd8/0x138
[ 326.503876] __report_bad_irq+0x58/0xe4
[ 326.503893] note_interrupt+0x2dc/0x3a0
[ 326.503913] handle_irq_event_percpu+0x90/0xa0
[ 326.503922] handle_irq_event+0x50/0xf0
[ 326.503932] handle_fasteoi_irq+0xc0/0x170
[ 326.503940] generic_handle_irq+0x40/0x60
[ 326.503947] __handle_domain_irq+0x70/0xd0
[ 326.503953] gic_handle_irq+0x68/0x134
[ 326.503957] el1_irq+0xd0/0x180
[ 326.503973] __bitmap_and+0x1c/0x80
[ 326.503983] rebalance_domains+0x298/0x3a0
[ 326.503989] run_rebalance_domains+0x54/0x80
[ 326.503993] __do_softirq+0x140/0x3e8
[ 326.504006] irq_exit+0xc0/0xe0
[ 326.504012] __handle_domain_irq+0x74/0xd0
[ 326.504016] gic_handle_irq+0x68/0x134
[ 326.504021] el1_irq+0xd0/0x180
[ 326.504038] tick_nohz_idle_exit+0x6c/0xc0
[ 326.504042] do_idle+0x188/0x270
[ 326.504047] cpu_startup_entry+0x30/0x70
[ 326.504059] rest_init+0xdc/0xe8
[ 326.504075] arch_call_rest_init+0x18/0x20
[ 326.504080] start_kernel+0x500/0x538
[ 326.504085] handlers:
[ 326.506438] [<0000000080b2cb9f>] tegra_pcie_rp_irq_handler threaded [<0000000072a6e636>] tegra_pcie_rp_irq_thread
[ 326.517023] [<00000000daf0649b>] pcie_pme_irq
[ 326.521517] [<00000000d940327e>] aer_irq threaded [<00000000445996a2>] aer_isr
[ 326.528966] Disabling IRQ #65
Subsequent runs of cat perf_test yield this in dmesg:
[ 1629.245230] tegra194-pcie 14100000.pcie: perf_test: DD WR, CH: 0 SZ: 267386880 B, time: 4795902405 ns
[ 1634.252759] tegra194-pcie 14100000.pcie: edma_submit_direct_rx: DD RD CH: 0 TO
[ 1634.260248] tegra194-pcie 14100000.pcie: perf_test: DD RD, SZ: 267386880 B CH: 0 failed
[ 1713.797575] tegra194-pcie 14100000.pcie: perf_test: DD WR, CH: 0 SZ: 267386880 B, time: 4819465563 ns
[ 1718.989272] tegra194-pcie 14100000.pcie: edma_submit_direct_rx: DD RD CH: 0 TO
[ 1718.996768] tegra194-pcie 14100000.pcie: perf_test: DD RD, SZ: 267386880 B CH: 0 failed
[ 1768.293684] tegra194-pcie 14100000.pcie: perf_test: DD WR, CH: 0 SZ: 267386880 B, time: 4769931990 ns
[ 1773.517746] tegra194-pcie 14100000.pcie: edma_submit_direct_rx: DD RD CH: 0 TO
[ 1773.525247] tegra194-pcie 14100000.pcie: perf_test: DD RD, SZ: 267386880 B CH: 0 failed