Geforce GTX 780 and GPUDirect RDMA

Accelerated Computing CUDA CUDA Programming and Performance
#1

Hello,

Does the Geforce GTX 780 (kepler) gpu support GPUDirect RDMA with a third party device (Mellanox network card)?

I have been looking for information about this but cant find a definitive answer. In some places it says that only Tesla cards support this features and in other places it says “Kepler-class GPU” which I think would include the GTX 780.

Thanks
/Björn

#2

As far as I know even the Titan cards offer no GPUDirect. You have to invest into a Tesla to get access to that feature.

#3

I can confirm the non-Tesla cards will return a “Device not supported” error if you try to use it.

#4

Thanks for the information.

#5

Does anyone know of a complete list showing which cards support which GPU-Direct functionality?

#6

I’ve requested such a list from support, but even support wasn’t able to help me. The best I could find is, if it says Tesla (since Kepler?), Yes, if anything else, No. Please correct me if other (cheaper) cards are also supported.

This seems to be a very frequently asked question:
https://devtalk.nvidia.com/default/topic/1038428/?comment=5276113

I’m also looking for more information on how to support GPUDirect RDMA/Async on the NIC side. What should be implemented etc. if anyone has information it would be much appreciated:
https://devtalk.nvidia.com/default/topic/1042508/other-libraries/gpudirect-rdma-async-for-dl-acceleration-mpi-nccl-/

#7

https://developer.nvidia.com/gpudirect

“…applications running on NVIDIA Tesla™ and Quadro™ products”.
“All you need is CUDA Toolkit v4.0 and R270 drivers (or later) and a system with two or more Fermi- or Kepler-architecture GPUs on the same PCIe bus”.

#8

Thank you. Here is another requirement:

“A recent CUDA Toolkit, minimally 8.0, because of the CUDA driver MemOP APIs.”

https://github.com/gpudirect/libgdsync

GPUDirect is a range of technologies. RDMA/Async for example is for GPUs on different nodes which has different requirements. It takes quite some digging to find the different nuances and requirements.