RDMA between K80 GPU and remote host

Is it possible to perform RDMA operations between a GPU’s memory and a remote host’s memory?

Looking at the documentation available online I wasn’t able to find examples or documentation that indicates this is possible – online documentation only talks about GPU-GPU communication.

Can anyone clarify if this is currently possible?



Yes, the peerDirect RDMA allows that.

You can perform RDMA reads/writes directly from/to GPU memory.

Please refer to the following user manual:

http://www.mellanox.com/related-docs/prod_software/Mellanox_GPUDirect_User_Manual_v1.3.pdf http://www.mellanox.com/related-docs/prod_software/Mellanox_GPUDirect_User_Manual_v1.3.pdf

For example, osu_latency can be used with “-d cuda D D” that means GPU to GPU or you can change it to “H H” (host to host) or to any other combination- “D H” or “H D”.