I’m building a new desktop computer for the purposes of VR game development and more importantly, developing machine learning software including all manner of neural nets and convolution. I’ll be using the usual libraries (Theano, Caffe, Tensorflow, Neon, DL4J, etc.). I’ve read that SLI is not good for this but that multiple GPUs are.
My first question is: Is SLI truly not suitable for machine learning even using the new SLI bridge?
Secondly: Assuming I purchase multiple GPUs (SLI or not), can I use these libraries to execute a task across them?
Finally: Which would be more desirable to start out with for this use-case… Two GTX 1080s or one Titan X Pascal?
I don’t knwo about the “new SLI bridge”, but traditionally the SLI bridge has been a low-bandwidth interconnect whose primary purpose was synchronization, not bulk transport. It’s a “graphics thing” that plays no role in CUDA programming.
The GPUs on boards with dual GPUs (e.g. K80), and multiple GPUs across different boards can communicate at high-speed using peer-to-peer communications across PCIe. Note that there are various restrictions and limitations, starting with the host system, so I would suggest studying NVIDIA’s documentation carefully before starting to build a rig with a massive amount of GPUs in them. I think consumer GPUs may not support peer-to-peer, but hopefully someone with practical experience shows up in this thread and can share with us some of the basic pre-requisites for successful peer-to-peer communication.
I am not familiar with any of the deep learning libraries, and therefore cannot comment on the quality of their multi-GPU support.