Hello all!
I would like to buy a motherboard that will have up to four PCI express graphic cards slots and set it up with four Tesla cards.
Do you have any suggestions about what motherboard to buy and if the whole implementation is worth the case ?
Thanks in advance!
I guess there aren’t many you can choose from. When you have four GPUs connected, usually bandwidth would be a concern because the bus lanes will be shared among all slots. But if you dont care about the bandwidth, I listed two mb that I know supporting four PCI-e, whether in PCI-e 1.0 or 2.0 or hybrid mode:
ASUS P5K64 WS
TYAN S2915
I have S2915 for myself, which can operate at 16x/8x/16x/8x, according the specification. However what I observed in cuda, after installing 4 GPUs, is that they’re all running at 8x speed (under Linux). Not sure if it’s my setup problem or not, but…just be aware of that potential problem…:P
I guess there should be some better motherboard for ultimate cuda cluster setup. Any suggestion? Thanks!
Mark
I think another option could be the Skulltrail MB from Intel. My question is that if the Skulltrail can run two 9800 gx2 card plus 2 8800 GT for a total of 6 CUDA devices.
This could fit nicely with the fact that the Skulltrail can accomodate 2 quad core xeon for a total of 8 cores. This monster could sustain a peak of 5TFLOPS for less than $10k. Anyone knows if this is feasible or not?
Crepatas
Wow, this one is cool!! However I remember I read it somewhere that cuda only supports up to 4 devices in current release, is that true?
Anyway, I think this should be THE ONE to build cluster with several TFlops!
Thanks for the info. External Media
Mark
I don’t know what is officially supported, but the current release does work with more than just 4 GPUs. One of the guys at Tycrid has our program VMD running with 6 GPUs using one of their PCIe backplanes:
Info) VMD for LINUXAMD64, version 1.8.7a19 (March 19, 2008)
Info) http://www.ks.uiuc.edu/Research/vmd/
Info) Email questions and bug reports to vmd@ks.uiuc.edu
Info) Please include this reference in published work using VMD:
Info) Humphrey, W., Dalke, A. and Schulten, K., `VMD - Visual
Info) Molecular Dynamics’, J. Molec. Graphics 1996, 14.1, 33-38.
Info) -------------------------------------------------------------
Info) Multithreading available, 8 CPUs detected.
Info) Free system memory: 3634MB (92%)
Info) Detected 6 CUDA accelerators:
Info) [0]: ‘Tesla C870’ 1.35 GHz, Compute 1.0, 1535MB RAM
Info) [1]: ‘Tesla C870’ 1.35 GHz, Compute 1.0, 1535MB RAM
Info) [2]: ‘Tesla C870’ 1.35 GHz, Compute 1.0, 1535MB RAM
Info) [3]: ‘Tesla C870’ 1.35 GHz, Compute 1.0, 1535MB RAM
Info) [4]: ‘Tesla C870’ 1.35 GHz, Compute 1.0, 1535MB RAM
Info) [5]: ‘Tesla C870’ 1.35 GHz, Compute 1.0, 1535MB RAM
Cheers,
John Stone
correct me if I am wrong, but you can’t physically fit more than two Tesla boards on any existing mobo (even skulltrail) because they are double wide and the bottom two PCI-e slots are adjacent (have to use the first slot for something w/ video out). The most you could fit is 4x single wide or 3x double wides. :blink:
So, this isn’t your standard MB Moulik :)
