Hi, I also am interested in using a system with multiple GTX985, so we can get in touch. First of all I am looking for a system with 2 GTX295 into a single workstation, but if possible also systems with 3-4 boards will be wellcome.
Can you tell me how have you manage to arrange 3 GTX295 into a single workstation?
Testing an X58 Intel system with three GTX280s at full power, we have never seen a system power draw of more than 800W. It appears to be difficult to reach full rated power consumption in GPGPU applications.
The real challenge of multi-GPU setups is not the PSU, but rather cooling!
I’m not sure if the two 30A rails for mainboard and CPU will deliver enough power to the PCIe bus. Keep in mind that those cards in total will draw up to 300W from the PCIe bus!
My recommendation (following the FASTRA recommendation) for a 4xGTX295 setup still is the Thermaltake ToughPower 1500W. My system has been running for the past two weeks non-stop at full load without a single issue.
Sorry, I don’t have the answer - I have another question instead. Can you tell the exact configuration with which it is possible to have 4 x GTX 295 in one box? We’re planning to buy these, but not sure about other hardware needed.
Right motherboard is the key – MSI K9A2 Platinum, having 4 PCI x16 slots with space in between to house double-fat cards. Power supply is another key - it should be 1250-1500 Watt unit with enough PCI-E connectors. Case must support 8 cards, not 7 as usual. And last, operating system should be either XP x32 or x64, not Vista, not Win 7. Driver compatibility is real issue.
I’m not sure what kind of motherboard you are using (MSI?), but for the Asus M3A79-T there is a BIOS entry for “GPP Slots Power Limit” which comes with the default value of 25W. I would guess the same option exists on all boards with a 790FX chipset.
The questions are:
Is this limit in BIOS effective? Does it really limit power draw from the motherboard?
Can a GTX295 run with only 25W from the motherboard? It appears that dual-GPU cards from ATI/AMD may need more.
This might be a problem (at least in theory). The spec sheet says the GTX 295 has a maximum power draw of 289W. The 6-pin PCI-E connector is rated to supply 75W, the 8-pin connector is rated for 150W, and the PCI-E slot itself provides another 75W. If the slot is limited to 25W, then the most that can be delivered to the card is 250W.
That said, I doubt the GTX 295 actually hits 289W in practice, but I don’t know what happens if you start to bump up against this limit.
Jaak - You’ve been a big help to us all and I want to thank you for that. I hear you on the dual PSU solution and that is what we do in the lab when running four GTX 295’s. Unfortunately, for use within enclosed cases with limited room it has to be a single PSU to fit inside the box. We would like to find a good single PSU solution that can run four GTX 295s.
The best so far appears to be the Enermax Jaak cites on his truly superb and useful web page, but it is only available in the US in a 1050W version. This apparently is exactly the same PSU, with a different AC power cord, that is sold as a 1250W unit in the UK. It is good for 1250W when run on European 220V power but is rated only at 1050W when run on lower US AC voltage.
So far we’ve not been able to find a 1250W to 1500W PSU in the US that has four six-pin and four eight-pin PCI-E cables. I think the problem is that it is difficult to get UL approval for the higher wattage PSUs in the US. Does anyone know of such a PSU?
The way power supplies (PSUs) work is that you plug them into the motherboard through the main 24 pin socket and then when you turn on the motherboard that tells the PSU to turn on. If you have two standard PC power supplies, only one of them is plugged into the motherboard. How do you get the other one to turn on? The solution is to have a little bridge adapter cable that brings out a second 24 pin socket into which the second PSU plugs into to make it think it is attached to the motherboard. The power on signal from the motherboard is the only wiring to that second socket.
You can wire that up yourself or buy pre-made units like this:
They work great. Power up the motherboard and both PSUs turn on.
By the way, on my other post I got some private emails suggesting the ENERMAX GALAXY EVO EGX1250EWT-01, at 1250W. This is not as efficient as the Revolution, but it seems to have a total of 8 6+2 PCI-E cables. Anyone using that?
From 80plus.org, it looks like some of the gold certified (90% efficiency) power supplies are going to start being sold in a few months; it seems like most are around the 750w range though (and the enermax revolution is already very good).
Since the power sockets in most US homes are only rated for 110-120V, 15A, reaching these high powers (after efficiency losses) is very hard to do while still staying compliant with the electrical codes here. The ToughPower and the Revolution appear to be 220V-240V only devices which are going to be hard to find in the US.
The Cooler Master does appear to be available here, although there are two models with almost the same name. If you just search for “Cooler Master Real Power Pro 1250W”, you will find some models with only 4x6-pin and 2x8-pin. You have to add “Quad-SLI” to the search in order to find the model with 6x6-pin and 3x8-pin. Interestingly, this PSU claims to draw 16A @ 115V, which sounds like it technically is over-spec for a standard US plug with parallel prongs. (20A sockets in the US usually take a plug with one prong turned 90 degrees.) This might explain why this PSU has certifications from a long list of electrical certification boards except one in the US.
Anyway, I’m hardly an electrician and not qualified to assess the practical safety of one of these devices plugged into US wiring. I just know my employer (an organization which is super-paranoid about such safety issues) would not allow me to use the Cooler Master because it lacks a UL certification in the US. (Figured I’d throw that out there in case anyone else has similar electrical safety standards at their workplace.)
I should point out that another option (besides moving to a 220V country, using two full PSUs on a tray, or slightly overloading one power socket) is to use a standard PSU to drive three cards, and then drive the 4th card with an auxiliary power supply like this one:
It installs into a 5.25" bay, and feeds its own power cord out one of the rear slots. This might be a better fit if you are building a quad GTX 295 inside a tower case, rather than on a rackmount tray. I believe there is a high-end Lian Li case with 10 rear slots, so you would have room both for the four cards and the power supply feed through plate. The Antec Nine Hundred Two (no idea why they insist on spelling it out) only has 8 slots, but appears to have two round cable ports on the back panel, so you could probably feed the second power cord directly through there.
(Disclaimer: I haven’t built any of these configurations… yet.)
I’ll report how it goes with that unit and four GTX 295 cards. We use very compact cases for travel so we don’t have any 5.25" slots at all in them.
One last thing: if the desire is not to overload an AC power socket, using a supplemental power supply like the 5.25" bay unit won’t help if both power supplies are plugged into the same circuit. You have to plug them into different circuits. That’s the same deal with any dual PSU solution. So far our engineers have been just plugging this stuff willy-nilly into whatever AC sockets happen to be around in the labs or their cubicles. That will have to change as everyone moves up to four card rigs. They’re already planning for it as part of a move to new, bigger facilities where all the engineering spaces will have massive numbers of independent 20A circuits, more cooling capacity in the HVAC, etc.