How to use RDMA write on DPA memory?

Infrastructure & Networking DOCA Getting Started & Resources
1

I allocated memory on the DPU’s Arm side, and then used RDMA write on this memory in DPA. This works.
The steps are as follows:
1、Allocate memory from the host
2、Create an mmap
3、Export the mmap
4、Create an mmap from the export
5、Get the mmap handle from the export mmap

However, when I change the first step to allocate memory from DPA instead (use doca_dpa_mem_alloc()), the program gets stuck after performing RDMA, waiting indefinitely for the RDMA write to complete. I modified my program based on the DPA_all_to_all sample.

May I ask, when allocating memory on the DPA, is it unnecessary to export it for remote RDMA access?

2

Hi 1563901725,

After allocating memory on the DPA, you shold

  1. Register or export the DPA memory for remote access using the appropriate DOCA API (e.g., doca_dpa_mem_dev_export() or similar, depending on the version and context).

  2. Obtain the remote key (rkey) or exported handle and share it with the RDMA initiator.

  3. Use this handle/rkey in the RDMA operation to ensure the remote side can access the DPA-allocated memory.

For details, you can refer to article: https://docs.nvidia.com/doca/archive/doca-v2.2.1/dpa-subsystem-programming-guide/index.html

Regards,

Quanying