AGX Orin Devkit PCIE bandwidth test

Hi,

We have DMA driver in kernel source but not enabled by default. To enable this,

  1. Enable the same by adding below lines in defconfig(arch/arm64/configs/defconfig) file

CONFIG_PCIE_EPF_DMA_TEST=y
CONFIG_TEGRA_PCIE_DMA_TEST=y

• First one enables PCIe End-Point function EDMA test framework to validate on EP side
• Second one enables PCIe EP driver on Root-Port to validate on RP side.

  1. Enable Jetson Orin EP side (this part is same as official document)

cd /sys/kernel/config/pci_ep/
mkdir functions/tegra_pcie_dma_epf/func1
echo 0x10DE > functions/tegra_pcie_dma_epf/func1/vendorid
echo 0x229a > functions/tegra_pcie_dma_epf/func1/deviceid
echo 16 > functions/tegra_pcie_dma_epf/func1/msi_interrupts
ln -s functions/tegra_pcie_dma_epf/func1 controllers/141a0000.pcie_ep/
echo 1 > controllers/141a0000.pcie_ep/start

  1. Before performing EDMA test

Framework:

Once Driver are enabled and platform is booted. Each driver creates its own debugfs directory.

For EP directory is: /sys/kernel/debug/<contorller_addr>.pcie_ep_epf_dma_test/
example: PCIe C6 EP controller can be referred at /sys/kernel/debug/141c0000.pcie_ep_epf_dma_test/

For RP directory is: /sys/kernel/debug/:01:00.0_pcie_dma_test/
example: PCIe C5 RP controller can be referred at /sys/kernel/debug/0005:01:00.0_pcie_dma_test/

Configuration:

Configurable parameters are referenced via files.

edma_ch → used for configuring number of EDMA channels and their modes. Bit definition is
• [0:3] - To set mode of RD/WR channels. 0-Sync, 1-Async
• [4-7] - To enable RD/WR channels. 0-Disable, 1-Enable
• 31 - is used to enable Remote EDMA mode
• 30 - is used to trigger ABORT use-case validations
• so value of 0xF1 means. all channels enabled for WR mode with channel 0 in async and rest of the
channel in sync mode.
• Note: During testing, If an async channel is selected first and then sync channel, high chances
that bandwidth is shared between these channels.
• For effective bandwidth Calculations, ensure that all channels are enabled in same mode
only(Sync/Async).

nents → number of descriptors to be populated in each DMA submission(tegra_pcie_edma_submit_xfer API
call).
• When more than one DMA channel is enabled, these nents are splitted for those many channels.
example : if nents = 2 and edma_ch = 0x3, each DMA channel gets one nent each
• Note: nents*dma_size cannot cross 127MB

dma_size → used to specify size in bytes to be transferred in each SW transaction.

stress_count → used for indicating how many number of SW transaction needs to scheduled in one
execution.

  1. Start to test DMA performance

Configure below settings of 16Mb size and single channel in async mode for 1000 iterations of 4 nents.

echo 16777216 > dma_size
echo 4 > nents
echo 1000 > stress_count
echo 0x11 > edma_ch
cat edmalib_test

If the test is done correctly, dmesg will have the bandwidth information.

1 Like