Doca_mmap_create_from_export segfaults when local mmaps are created before doca_rdma_create

Infrastructure & Networking Software And Drivers RDMA Software For GPU
1

I get a segmentation fault in doca_mmap_create_from_export() that depends on the ordering of DOCA API calls. This occurs when local memory maps (doca_mmap) are created and started on a device before doca_rdma_create() is called for the same device

DOCA SDK Version: 3.2.0
Platform: Linux amd64, ConnectX-6 DX with GPU RTX5000 Ada

Working order (matches DOCA samples):

  1. doca_gpu_create()

  2. doca_rdma_create() + doca_ctx_start()

  3. doca_gpu_mem_alloc() + doca_mmap_create() / doca_mmap_set_dmabuf_memrange() / doca_mmap_add_dev() / doca_mmap_start() (local mmaps)

  4. doca_mmap_export_rdma() (export local descriptor)

  5. doca_mmap_create_from_export() (import remote descriptor) — works

Failing order:

  1. doca_gpu_create()

  2. doca_gpu_mem_alloc() + doca_mmap_create() / doca_mmap_set_dmabuf_memrange() / doca_mmap_add_dev() / doca_mmap_start() (local mmaps)

  3. doca_mmap_export_rdma() (export local descriptor)

  4. doca_rdma_create() + doca_ctx_start()

  5. doca_mmap_create_from_export() (import remote descriptor) — segfault

Details:

  • All doca_mmap and doca_rdma operations use the same doca_dev

  • The GPU memory is mapped using dmabuf (doca_mmap_set_dmabuf_memrange)

  • The exported descriptor is valid (124 bytes, transferred via TCP out-of-band channel)

  • The doca_dev pointer is valid and non-null at the time of the crash

  • Creating 1 local mmap before doca_rdma_create sometimes works; creating 2 local mmaps (for different memory ranges) reliably triggers the segfault

  • All doca_mmap_* calls return DOCA_SUCCESS — no errors reported before the crash

  • The remote side (server) successfully performs doca_mmap_create_from_export with the same code pattern

Reproduction:

Both endpoints share the same GPU (doca_gpu_create with same PCIe BDF) but use different IB devices (e.g., mlx5_0 for server, mlx5_2 for client). RDMA connection (RC transport) is fully established before the failing doca_mmap_create_from_export call

Question:

Is there a requirement that doca_rdma_create() must be called before doca_mmap_create() / doca_mmap_start()? The DOCA programming guide and API reference don’t mention this ordering constraint. If this is a known requirement, it would be helpful to have it documented