[*]CPU Intel E8500 (2 cores @ 3.16 GHz and 6 MB cache)[*]8 GB DDR3 RAM @ 1333 MHz[*]Mobo Ultra 790i[*]3 graphics cards 9800 GX2 (1 EVGA overclocked by the manufacturer, 1 Nvidia, 1 XFX)[*]PSU Cooler Master Silent Pro Gold 1000 W[*]Windows 7 Ultimate 64 bit[*]CUDA toolkit 3.2[*]Driver Nvidia 266.58[*]Visual Studio 2008 with 64 bit support
Windows sees correctly 6 GPUs with the right driver. Nvidia system monitor shows the physical state of all the six GPUs. But if I try to launch a CUDA application from a command window to enumerate the GPUs I can see just a single GPU when the multi-GPU flag is disabled in the Nvidia performance monitor. When I enable the flag to a multi-gpu configuration, the card with the connected monitor configures to multi-GPU, CUDA works correctly and I can see 5 GPUs or 4 GPUs, apparently in a random way after the boot of the machine. Has anybody out there experienced such a behavior? Could there be an explanation or a possible correction to it to have all the GPUs at work with CUDA applications?
The first analysis step: what does deviceQuery from the SDK say? That’s dead simple.
Some apps don’t necessarily list all GPUs… your app may only be listing display GPUs for some reason.
Also, the multi-GPU flag is irrelevant for CUDA… that’s only used for graphics acceleration. It doesn’t matter what it’s set to. (It USED to matter in the CUDA 1.0 days but not for a long time now)
I have run deviceQuery from the SDK and I get 4 GPUs instead of the expected 6. My application gives the same. Currently I have the multi-GPU flag on. I cited this because, when I turn it off, I experience a different behavior with respect to CUDA: I see just one GPU.
Of course, I can happily work with 4 GPUs but I would like to get the full power at work if I can as I do some extensive lattice computations.
I cannot try that as I have just a single monitor but I have the mouse pointer that goes out of the screen as there would be more monitors. There is a strange behavior of the machine as now I just see 4 GPUs from a CUDA application (Windows always sees 6 GPUs as Nvidia monitor). Now, if I disable multi-GPU mode and I restart my machine I will turn to see a single GPU. Then, enabling again the multi-GPU mode at the next boot I will see immediately 5 GPUs but this will not be stable as, rebooting again, it will fall down to 4 GPUs.
I was aware of this kind of problems with previous Nvidia drivers but I was sure it was overcome. On the other hand, people working with a large number of CUDA devices there should be a significant number by now. So I cannot exclude that there could be a glitch that, once fixed, could settle all the matter.