The new dual-GPU GTX590 is still hard to find, but I did finally get one, and I’ll be ordering 4-5 more. My software scales well over multi-GPU, and it’s worked well with the previous GT200 dual GTX295. The specs of the GTX590 are also known, so I didn’t have any doubt about performance or compatability.
The biggest question I did have was about power use… the GTX590 has a hardware PCB design flaw which makes it extremely easy to fry itself. This is typically only when the crazy kids try to crank voltages too high, but the GTX590 is notable about its unprecedentedly high wattage and failure rate.
Still, as I’ve posted about before, we CUDA guys don’t really need to worry. CUDA apps use significantly lower wattage than graphics apps, so there should be plenty of wattage headroom.
The exact wattage used in CUDA apps varies of course, but as a rough approximation, a fully loaded CUDA app would use about 65% of the wattage of a fully loaded graphics apps. All reviews you ever read will be for and by the graphics guys who are optimizing frames-per-second of the latest eye-candy shooting games.
Still, those fried card reports show the GTX590 is likely the least robust GPU to use. Is that OK for CUDA? Do we actually reach any harsh limits?
Well, that’s what I measured of course. I have a system power meter (a Kill-O-Watt) so I can measure instantaneous full-system wall socket power draw. This doesn’t give you exact GPU wattages, but in some ways it’s even more useful since you can tell if your PSU is overloaded or whatever.
My benchmark (for both power and speed) is one of my own applications… a Monte Carlo integral equation computation over a large geometric database. One GTX480 runs my GPU app about 25 times faster than a optimized one-core CPU version. The computation usually takes many hours to run on the GPU, and my code is multi-GPU aware so it’s common to run 3 GPUs for 4 or 8 hour runs straight. The code is entirely GPU based so the CPU is idle during computation (other than dealing with zero-copy memory and minor kernel scheduling).
This is pretty much the most severe app I can run in CUDA… the GPU is going full speed for hours (or days!) at a time with no rest or mercy.
I’m testing on Linux (Ubuntu 10.10) using an older driver, 260.19.36. There are newer drivers, but these affect my video pretty badly, so I stuck with what worked. That said, there were some issues with the GPUs staying above idle rate long after computes have finished… those may or may not be fixed in future driver updates. But for wattage use at idle and at full load, I don’t expect the values to change. (There’s strong reason to use a more updated driver for power limiting, but again that’s for the crazy Furmark guys who like to stress VRMs)
My system specs shouldn’t matter too much, since we’ll be comparing the same machine after swapping out a single GTX480 with the new GTX590.
The motherboard is a ASUS P7T Revolution, the CPU is an i7-980X, 12GB of RAM. There is a GT240 GPU for display only.
In the first configuration, I’ll show a GTX480, then I swap to the GTX590 and run with one of the GTX590 GPUs active and then both active.
Wattages are measured from the wall socket, and therefore actual power use is lower (the PSU likely is 15% inefficient).
Remember these are wattages for the entire system, so it includes CPU, drives, display card, everything. But we can easily see the GPU differences during load (where the GPU is the only change in the system load.)
Wattage of the GPUs typically increases with time, likely from the GPUs heating up… the GPUs themselves seem to use more power when hot (and the fan likely uses a few watts).
Base run. GTX 480.
Initial idle: 181 watts. GPU at 37 degrees C.
At application start: 308 Watts. Temp 39 degrees C.
At application end: 331 Watts. Temp 66 degrees C.
GPU run time: 502 seconds.
GTX 590, using just one of the onboard GPUs.
Initial idle: 186 watts. GPUs at 36 and 34 degrees.
Application start: 285 Watts. 40 and 34 degrees.
Application end: 310 Watts. 77 and 36 degrees.
Run time: 542 seconds
GTX 590, using both onboard GPUs simultaneously
Idle: 186 watts. GPUs at 36 and 34 degrees.
Application start: 400 Watts. 40 and 40 degrees.
Application end: 428 Watts. 80 and 75 degrees.
Run time: 272 seconds
Speed wise, the GTX590 is just as fast in CUDA as expected from its core count and clock rates.
The GTX480 has 480 cores at 1.4GHz, the GTX 590 has 2 GPUs each at 512 cores and 1.26 GHz (for the EVGA Classified GTX590 I have).
The run times I got show the same ratio… the GTX590 is about 7% slower than the GTX480 (from cores alone you’d expect 5% slower, but the GTX590’s RAM is slower too.)
As expected, using dual GPUs of the GTX590 is twice as fast as a single GPU. That’s more an measure of my CUDA code’s efficiency, but it’s good to see there’s no unexpected hardware slowdown.
These speed measurements are not a surprise.
Power use was the big unknown and the results are quite pleasant. We CUDA folk can smile and feel smug.
Idle power wasn’t measured directly, but it’s still quite low, only 5 Watts more than a GTX480. Loaded, using a single GPU of the GTX590 consumes roughly the same wattage as the GTX480.
Using both GPUs at full stress uses about 242 Watts more than the card than idle. Assuming idle is about 30 Watts (I did not measure this) it means the full GTX590 is using about 275 Watts at full load when running both GPUs. Even this value is too large since that’s wall socket power, so actual GPU use is likely around 250 watts at full load. This is a very very large and comfortable margin compared to the 375 Watt design limit given by card’s maximum PCIE power input.
Summary: The dual GTX 590 works in CUDA as expected. It’s great. For CUDA apps the card won’t use more than 275 Watts even for intense and sustained loads.