Xavier pcie share memory access too slow

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier
,
1

There are two xavier modules on our board. They previously communicated through PCIE. The way of pcie communication is shared memory. The shared memory is allocated on CMA. But after our test, the reading and writing speed of pcie shared memory is very slow. The test results are as follows:
pcie WRITE speed 402.085699MB/S,write time: 310.879000 ms
pcie READ speed 22.136886MB/S,read time: 5646.684000 ms

pcie WRITE speed 404.777016MB/S,write time: 308.812000 ms
pcie READ speed 13.173793MB/S,read time: 9488.535000 ms

pcie WRITE speed 404.960605MB/S,write time: 308.672000 ms
pcie READ speed 12.682657MB/S,read time: 9855.979000 ms

pcie WRITE speed 403.574703MB/S,write time: 309.732000 ms
pcie READ speed 17.510679MB/S,read time: 7138.501000 ms

pcie WRITE speed 403.267434MB/S,write time: 309.968000 ms
pcie READ speed 85.927484MB/S,read time: 1454.715000 ms

pcie WRITE speed 404.110940MB/S,write time: 309.321000 ms
pcie READ speed 13.202946MB/S,read time: 9467.584000 ms

pcie WRITE speed 404.124005MB/S,write time: 309.311000 ms
pcie READ speed 12.679055MB/S,read time: 9858.779000 ms

How can we solve this problem

2

two xavier modules on one board?

4

yes

5

Please provide more details on the tests.
Are you using DMA for this ?

6

We used DMA，We use the sample driver provided by nvidia, and then change the size of shared memory from 1M to 256M, using CMA memory

7

Try lspci -vvv and check width used (mostly 8 or 16) and the speed is 8 GTs.

Check whether intterupt is enabled.

Check whether any batch kind of DMA submission is possible.
I have not seen the sample driver

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.