Does GPUDirect RDMA support CUDA/OpenGL Interop Buffers?"

Accelerated Computing CUDA CUDA Programming and Performance
1

Hi,
I am trying to transfer VBO data rendered in OpenGL using GPUDirect RDMA with nvidia-peermem driver.

Referring to the simpleGL from CUDA sample here,
when I register the CUDA buffer obtained through OpenGL/CUDA interop(In the sample code, it is dptr pointer) using rdma_reg_read() from librdmacm, I get a ‘Bad address’ error.

I tried another approach…
First, I allocated a region using cudaMalloc(), registered it with rdma_reg_read(), and copied the data of the CUDA buffer obtained through OpenGL/CUDA interop using cudaMemcpy(cudaMemcpyDeviceToDevice). This method succeeds.

My question is whether there are any restrictions or limitations on the CUDA buffer that can be registered with GPUDirect RDMA.

2

Platform:

# cat /etc/redhat-release
Rocky Linux release 8.6 (Green Obsidian)
# uname -a
Linux tower11 4.18.0-372.32.1.el8_6.x86_64 #1 SMP Thu Oct 27 15:18:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
# nvidia-smi
Thu Mar 13 12:40:58 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 560.35.05              Driver Version: 560.35.05      CUDA Version: 12.6     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA RTX 4500 Ada Gene...    Off |   00000000:47:00.0 Off |                  Off |
| 30%   31C    P8             19W /  210W |     242MiB /  24570MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A      1982      G   /usr/libexec/Xorg                              51MiB |
|    0   N/A  N/A      2158      G   /usr/bin/gnome-shell                            3MiB |
|    0   N/A  N/A   1004229      G   /usr/bin/gnome-shell                          167MiB |
+-----------------------------------------------------------------------------------------+

# ofed_info -s
MLNX_OFED_LINUX-24.10-1.1.4.0:

# ibv_devinfo
hca_id: mlx5_0
        transport:                      InfiniBand (0)
        fw_ver:                         16.35.4030
        node_guid:                      e8eb:d303:00fe:24a0
        sys_image_guid:                 e8eb:d303:00fe:24a0
        vendor_id:                      0x02c9
        vendor_part_id:                 4119
        hw_ver:                         0x0
        board_id:                       MT_0000000011
        phys_port_cnt:                  1
                port:   1
                        state:                  PORT_ACTIVE (4)
                        max_mtu:                4096 (5)
                        active_mtu:             1024 (3)
                        sm_lid:                 0
                        port_lid:               0
                        port_lmc:               0x00
                        link_layer:             Ethernet