Prepare devices on EP and RP side: [EP]
a) Edit the jetson-xavier.conf file and change ODMDATA to: ODMDATA="0x09191000";
b) Apply the patch and re-build the kernel Image
c) Replace your rebuild kernel Image to JetPack-5.1.2/Linux_for_Tegra/kernel/Image
d) Full flash the image: $ sudo ./flash.sh jetson-xavier mmcblk0p1
[RP]
Using default JetPack-5.1.2 ODMDATA to flash image on RP Xavier.
Connect two system using the appropriate PCIe cable
The steps 1 to 6 are done.
The only difference is step 7, I disabled the NetworkManagement service and installed Netplan to replace it. Which is mentioned by WayneWWW before.
I just checked some history test result. The performance of PCIe-vnet on your side is expected result as our side.
If you want to check the real DMA read/write which is not based on Vnet, then it could be faster.
But both cases have write performance faster than read perf.
The read and write operation main difference is Non-Posted and Posted Transactions
i)Non-posted transactions are requests that return a Completion TLP from the Completer, indicating that the transaction was successfully processed.
Memory Read is Non-posted ,For read requests, the Completer returns a Completion with Data TLP.
Memory read transactions use the split-completion model. The requester sends the read TLP. After the completer fetches the data from memory, it sends the data back in the form of
Completion with Data TLPs.
ii)Memory Writes are posted transactions. These transactions do not require a Completion TLP from the packet’s ultimate destination.
In the below procedure, x being the number of the root port controller whose DMA is being used for perf checkout
Write:
Go to the debugfs directory of the root port controller
cd /sys/kernel/debug/pcie-x/
Set channel number (set it to one of 0,1,2,3)
echo 1 > channel
3 Set size to 512MB
echo 0x20000000 > size
Set source address for DMA.
For this, grep for the string “—> Allocated memory for DMA” in dmesg log and use whatever address comes up in the grep output
Note: - don’t forget to replace 0xA0000000 with your grep output value. In case it is not found in grep output, save full kernel boot log and search in it
Set destination address for DMA
For this, execute the following command
lspci -vv | grep -i “region 0”
an example output would be something like
Region 0: Memory at 1f40000000 (32-bit, non-prefetchable) [size=512M]
So, use 1f40000000 as destination address
echo 0x1f40000000 > dst
Note: - don’t forget to replace 0x1f40000000 with your grep output value. In case it is not found in grep output, save full kernel boot log and search in it
Execute write test
cat write
It prints the output in the following format(use ‘dmesg |tail’ to get the output)
