Does Mellanox support DirectGMA?
DirectGMA is AMD’s PCIe peer-to-peer mechanism which is similar to Nvidia’s GPUDirect.
The one reference I could find was in a Mellanox slide presentation by Todd Wilde entitled "Paving The Road to Exascale Computing Paving The Road to Exascale Computing HPC Technology Update - PDF Free Download ":
“AMD DirectGMA P2P: Peer-to-Peer GPU-to-HCA data path under development”
Otherwise there is seemingly no mention of DirectGMA support.
Excellent suggestion, thank you. That does indeed look like the way to go.
I’m not a big expert on this, but I know the way to go today is to work with the ROCm driver, which supports Peer to Peer GPU VIA RDMA and has a peer plugin.
It’s not exactly DirectGMA (which never worked well beyond SD-I/O functionality).
See more details in - ROCm, a New Era in GPU Computing https://rocm.github.io/ , GitHub - rocmarchive/ROCnRDMA: ROCm Driver RDMA Peer to Peer Support GitHub - rocmarchive/ROCnRDMA: ROCm Driver RDMA Peer to Peer Support and Redirecting to https://rocm-documentation.readthedocs.io/en/latest/ Redirecting to https://rocm-documentation.readthedocs.io/en/latest/
Hope it helps.