When allocating memory with cuMemCreate, attempting to set SYNC_MEMOPS returns 801 (operation not supported). In my case, the buffer has gpuDirectRDMACapable set (both dependent GPU_DIRECT_RDMA device attributes are present), and this buffer is the target of GPU Direct RMDA operations from a third-party NIC, as well as host<->device copies, and kernel access.
The CUDA GDR recommendations, section 2.7, state that SYNC_MEMOPS is required for general GDR use cases.
Seems to works fine with cuMemAlloc.
What are the guidelines for buffers allocated with the VMM API, if SYNC_MEMOPS is not supported? Are there best practices for manual synchronization in the absence of SYNC_MEMOPS? Is synchronization fundamentally at odds with the VMM API in some way?
Any clarification on the overall synchronization guidelines and limitations w.r.t. the VMM API would be appreciated.
CUDA 12.5, Ice Lake, A100