Using Surface Memory as RDMA Buffers with GPUDirect RDMA

Accelerated Computing CUDA CUDA Programming and Performance
1

I am currently working with GPUDirect RDMA and have created a test application to evaluate its performance. The application works fine when I use a buffer allocated with cuMemAlloc. Here are the steps I followed successfully:

  1. Allocate memory using cuMemAlloc.
  2. Set pointer attribute using cuPointerSetAttribute.
  3. Pin the memory.
  4. Perform RDMA operations.
  5. Unpin the memory.

Now, I want to use surface memory (bound to arrays) as RDMA buffers. However, I’m facing issues with this approach. Here are the steps I attempted:

  1. Allocate memory using cudaMallocArray.
  2. Bind the allocated array to a surface.
  3. Attempt to set the pointer attribute with cuPointerSetAttribute (this step fails as it requires a CUdeviceptr).
  4. Pin the memory.
  5. Perform RDMA operations.
  6. Unpin the memory.

It seems that cuPointerSetAttribute does not accept memory allocated with cudaMallocArray as it requires a CUdeviceptr.

My Question: Is it possible to use surface memory (bound to arrays) as RDMA buffers, or only CUdeviceptr buffers allowed for RDMA operations with GPUDirect RDMA? If it’s possible, what is the correct procedure to set this up?

Thank you for your assistance!