CUDA development cluster (using old filing cabinet!) Advice needed on hardware specification

I second that. PSUs are the one area you can’t cut corners with. I have had favorable experiences with both Cheiftec and Corsair PSUs driver single and dual GT200/Fermi setups under 24/7 conditions.

Agreed on the PSU. I usually use Corsair or Antec power supplies.

The first PSU I ever bought for GPU usage was a highly discounted Rosewill that went bang a foot from my face (no debris, thankfully) when an electrolytic capacitor exploded a few minutes after turning it on. I happened to be on the phone with someone, who thought I had been shot in my apartment somehow. :)

Agreed on the PSU. I usually use Corsair or Antec power supplies.

The first PSU I ever bought for GPU usage was a highly discounted Rosewill that went bang a foot from my face (no debris, thankfully) when an electrolytic capacitor exploded a few minutes after turning it on. I happened to be on the phone with someone, who thought I had been shot in my apartment somehow. :)

Another vote for solid stable power from me. The other recommendations in this thread are all good; additionally, Tom’s Hardware just used a SeaSonic in a recent high end gaming build, here’s the link:

http://www.tomshardware.com/reviews/newegg…are,2753-4.html

Seventeam power supplies have also received good reviews at NewEgg. But I have no direct experience with either of these brands.

In general, we have a lot of the same issues as people building high powered gaming rigs (especially the need for powerful GPUs,) and many of their solutions are also good ones for us. Just don’t forget to think about all the details - will you have enough power on the appropriate rail or rails for what you’re powering; will you have enough PCI-e power cords, etc.

Regards,

Martin

Another vote for solid stable power from me. The other recommendations in this thread are all good; additionally, Tom’s Hardware just used a SeaSonic in a recent high end gaming build, here’s the link:

http://www.tomshardware.com/reviews/newegg…are,2753-4.html

Seventeam power supplies have also received good reviews at NewEgg. But I have no direct experience with either of these brands.

In general, we have a lot of the same issues as people building high powered gaming rigs (especially the need for powerful GPUs,) and many of their solutions are also good ones for us. Just don’t forget to think about all the details - will you have enough power on the appropriate rail or rails for what you’re powering; will you have enough PCI-e power cords, etc.

Regards,

Martin

Ok then, warning heeded! :)

Looks like the 750W Corsair PSUs have had some great reviews and cost about £100 so will have to do some rethinking about the spec or try and find some extra funds from somewhere. Would like a solid system to build from at the start so as you all say, no point in risking my hardware or spurious results if I scrimp on the power supplies.

Am I glad I posted on this site! Had some great feedback and has been very worthwhile.

Thanks again for all the tips guys.

Dan

Ok then, warning heeded! :)

Looks like the 750W Corsair PSUs have had some great reviews and cost about £100 so will have to do some rethinking about the spec or try and find some extra funds from somewhere. Would like a solid system to build from at the start so as you all say, no point in risking my hardware or spurious results if I scrimp on the power supplies.

Am I glad I posted on this site! Had some great feedback and has been very worthwhile.

Thanks again for all the tips guys.

Dan

To make sure I come under budget I’ve revised my spec and gone for a complete AMD system with Antec 750W PSUs. I’ll have to save the Intel route for later (if and when I can afford it!).

The spec thus far is:
4 x AMD Athlon 635, 4 x ASUS M4A88TD-V EVO ATX, 4 x 250GB, 4 x 2GB 1333MHz DDR 3 SDRAM, 4 x GTX 470, 4 x Antec 750W PSU

Total cost (through Insight) is £2664.60/$4208.84 so just over budget. Hopefully should be able to power 2 GTX 470 cards on my chosen boards with the Antec PSUs and get some decent performance.

To make sure I come under budget I’ve revised my spec and gone for a complete AMD system with Antec 750W PSUs. I’ll have to save the Intel route for later (if and when I can afford it!).

The spec thus far is:
4 x AMD Athlon 635, 4 x ASUS M4A88TD-V EVO ATX, 4 x 250GB, 4 x 2GB 1333MHz DDR 3 SDRAM, 4 x GTX 470, 4 x Antec 750W PSU

Total cost (through Insight) is £2664.60/$4208.84 so just over budget. Hopefully should be able to power 2 GTX 470 cards on my chosen boards with the Antec PSUs and get some decent performance.

I would be buying more, slower ram per CPU. With a 1280Mb GPU, you probably want double that in free memory per node, after the OS, application and support libs are loaded. I would have thought 4Gb memory per node, minimum.

I would be buying more, slower ram per CPU. With a 1280Mb GPU, you probably want double that in free memory per node, after the OS, application and support libs are loaded. I would have thought 4Gb memory per node, minimum.

Point noted avidday. Will see if I can stretch the budget to cover some additional RAM. Not much difference in price between the 1333MHz and 1066MHz RAM from Insight (in fact the 1066MHz is slightly more expensive!).

Based on your comment, 4GB per node for running a single GPU sounds much more reasonable and would give me 8GB per node when I run two GPUs per node too.

Point noted avidday. Will see if I can stretch the budget to cover some additional RAM. Not much difference in price between the 1333MHz and 1066MHz RAM from Insight (in fact the 1066MHz is slightly more expensive!).

Based on your comment, 4GB per node for running a single GPU sounds much more reasonable and would give me 8GB per node when I run two GPUs per node too.

Hi

maybe you should scrap the idea of buying 4 nodes, and only take 2 with 2 GPUs each. You save the money for half the CPUs, PSUs and MoBos which you should put into more RAM, better CPU and better MoBo on each node. For example the MoBo you choose (ASUS M4A88TD-V EVO ATX) only provides one full PCIe2.0 x16 slot. The second one is only connected with 4 lanes (one fourth of the bandwidth) which would definately cripple transfer rates if you ever think of putting a second GPU in.

This for example might be a better choice: Asus Crosshair IV Formula. While it costs much more per node it provides 4 x16 slots which can be used with 16 16 8 4 if I am not misinformed. Unfortunately it has no onboard GPU so you might want to add a GPU for graphics output (maybe öater). Best would be one from NVIDIA as well since it minimizes problems in assigning which GPU to use for output and which for compute. Think of something like a GT220 or a 9400GT. It should be actively cooled (has a fan) though. You might need to take a 850W power supply for that (which should not be much more expansive than the 750W) . I am convinced that such a solution would be much better than trying to squeeze 4nodes out of your budget and would give you the same computational power on GPUs.

Cheers

Ceearem

Hi

maybe you should scrap the idea of buying 4 nodes, and only take 2 with 2 GPUs each. You save the money for half the CPUs, PSUs and MoBos which you should put into more RAM, better CPU and better MoBo on each node. For example the MoBo you choose (ASUS M4A88TD-V EVO ATX) only provides one full PCIe2.0 x16 slot. The second one is only connected with 4 lanes (one fourth of the bandwidth) which would definately cripple transfer rates if you ever think of putting a second GPU in.

This for example might be a better choice: Asus Crosshair IV Formula. While it costs much more per node it provides 4 x16 slots which can be used with 16 16 8 4 if I am not misinformed. Unfortunately it has no onboard GPU so you might want to add a GPU for graphics output (maybe öater). Best would be one from NVIDIA as well since it minimizes problems in assigning which GPU to use for output and which for compute. Think of something like a GT220 or a 9400GT. It should be actively cooled (has a fan) though. You might need to take a 850W power supply for that (which should not be much more expansive than the 750W) . I am convinced that such a solution would be much better than trying to squeeze 4nodes out of your budget and would give you the same computational power on GPUs.

Cheers

Ceearem

Hey Ceearem,

Thanks for this. I mistakenly presumed that 2 x16 pcie slots meant that they both operate at x16 bandwidth. You might have guessed I’m quite new to this…

I’m struggling to find a motherboard with 2 full pcie x16 slots in my budget so have taken on board your suggestions. By cutting down to 2 nodes and using the Crosshair boards I can get the following for £2453/$3886 (i.e. just under budget!):

2 x AMD Phenom x4 955, 2 x ASUS Crosshair IV Formula, 2 x 250GB, 4 x 4GB 1333MHz DDR3 SDRAM, 4 x GTX 470, 2 x Antec 850W PSU, 2 x GeForce GT220

I’m sacrificing on the number of nodes but it seems this is the way to go given my budget. I’ll still have the 8 CPU cores but will have a decent GPU setup to play with. As you say, I have been able to upgrade my processor from an Athlon to a Phenom and can also now afford 8GB RAM per node.

Any other comments?

Hey Ceearem,

Thanks for this. I mistakenly presumed that 2 x16 pcie slots meant that they both operate at x16 bandwidth. You might have guessed I’m quite new to this…

I’m struggling to find a motherboard with 2 full pcie x16 slots in my budget so have taken on board your suggestions. By cutting down to 2 nodes and using the Crosshair boards I can get the following for £2453/$3886 (i.e. just under budget!):

2 x AMD Phenom x4 955, 2 x ASUS Crosshair IV Formula, 2 x 250GB, 4 x 4GB 1333MHz DDR3 SDRAM, 4 x GTX 470, 2 x Antec 850W PSU, 2 x GeForce GT220

I’m sacrificing on the number of nodes but it seems this is the way to go given my budget. I’ll still have the 8 CPU cores but will have a decent GPU setup to play with. As you say, I have been able to upgrade my processor from an Athlon to a Phenom and can also now afford 8GB RAM per node.

Any other comments?

The MSI 980i-G65 is the miracle bang-for-buck CUDA motherboard. It has built in NV video, so no display card is needed. It hosts THREE double width cards at x16 x16 x8 speeds. It’s cheap!


MSI 980i-G65
$158
Phenom 6-core CPU $200
4 GB DDR3 RAM $90
500GB Hard Drive $50

DVD Drive
$19
Antec p183 case $155
CP850 PSU $120
2 GTX470 $295

Total: $1382.

So you can build three of these dual GTX470 systems with your budget.


Alternatively, change case and PSU to:

Silverstone FT02 $239 (The ideal multi-GPU case!)
Silverstone 1000W PSU $183
3 GTX470 $295

Then you can buy two systems with 3 GPU each for $1824 each… comfortably under your budget.

I recommend the latter… but it’s all up to what you need.

The MSI 980i-G65 is the miracle bang-for-buck CUDA motherboard. It has built in NV video, so no display card is needed. It hosts THREE double width cards at x16 x16 x8 speeds. It’s cheap!


MSI 980i-G65
$158
Phenom 6-core CPU $200
4 GB DDR3 RAM $90
500GB Hard Drive $50

DVD Drive
$19
Antec p183 case $155
CP850 PSU $120
2 GTX470 $295

Total: $1382.

So you can build three of these dual GTX470 systems with your budget.


Alternatively, change case and PSU to:

Silverstone FT02 $239 (The ideal multi-GPU case!)
Silverstone 1000W PSU $183
3 GTX470 $295

Then you can buy two systems with 3 GPU each for $1824 each… comfortably under your budget.

I recommend the latter… but it’s all up to what you need.