I’m posting this with the aim to help others out there who may want to build their own personal supercomputer.
I’ve recently done it, I found very little advice on the net, I had a few problems so I’m putting this here.
My first port of call was here… [url=“http://www.nvidia.com/docs/IO/63567/web_DIY_PDF.pdf”]Page Not Found | NVIDIA
I wanted 4 tesla’s in one machine so I went for the Tyan s7025 motherboard. Unfortunately this board can’t actually take 4 double slot cards as advertised as the ports for the hard drive cables and so on are in the way.
The solution is to get some right angle SATA cables… [url=“http://www.amazon.co.uk/gp/product/B0001Y8UCU/ref=pd_lpo_k2_dp_sr_1?pf_rd_p=103612307&pf_rd_s=lpo-top-stripe&pf_rd_t=201&pf_rd_i=B0001Y8UI4&pf_rd_m=A3P5ROKL5A1OLE&pf_rd_r=0B4MX0FTMKF2EWNGF2GD”]http://www.amazon.co.uk/gp/product/B0001Y8...0FTMKF2EWNGF2GD[/url]
You will need 2 or 3 per machine (more if you have more than 1 HD)
We also tried PCIe extension leads but they didn’t work!
Also go into the PCIe settings in the BIOS and enable power management and set the PICe speed to 248. Having done this it works with 4 cards!!!
Fortunately I went for the Lian-Li case (which is excellent) as this case has PCI slots well below the reach of the motherboard, so with the extension cable in place the 4th card can easily be mounted there without any case modification. Other cases might also be roomy enough, I don’t know.
Next problem is the display, the Tyan s7025 motherboard has a built in Aspeed AST2050 graphics chip, so with 4 tesla’s (which have no display output) you can still see whats going on. However the onboard graphics are designed for server use, i.e. occasional plugging in a monitor to set things up, so they are essentially a glorified frame buffer… i.e. expect huge lag. More importantly it doesn’t do openGL, which means all the nice pretty CUDA demo’s won’t work and you may have issues setting up GUI’s (though not necessarily). If you want remote access, and don’t need fancy graphics output then its probably fine.
(EDIT: the ast2050 does do openGL (very slowly) but only if you haven’t installed the NVIDIA driver, once thats in the system there is no openGL)
Our solution was to buy a GTX285 (very similar spec to the Tesla C1060 and cheeper too! as other threads here will confirm) and use that as a 4th card rather than the Tesla.
This card works well and slightly faster than the tesla (it’s an over-clocked one so thats probably why)
We also had serious problems getting the Tesla cards to work as the onboard graphics is a different driver. I’ve attached a .pdf with a guide to installing them on this setup… You can get the scripts from my college Martin as mentioned in the .pdf if you need them.
UPDATE, you don’t need the scripts if you use the GTX285 with the Teslas and disable the onboard graphics in the BIOS.
So we now have a fully working system!!
Tesla_Installation_Guide.pdf (219 KB)