Problem with 3-GPU system


we want to use a 8600GT as the primary display adapter and two 8800GT as CUDA computation devices

we have the following problem:
the system automatically lowers the clocks of the 8800GTs to 399/399/810Mhz and 601/499/1512Mhz instead of leaving them at the default factory settings, only the 8600GT runs at default clock settings

we have tried several tuning applications (nTune, rivaTuner, Powerstrip) but all of them allow to change the clock only for the GPU attached to the display

our specifications:

  • Windows XP 64-bit
  • latest motherboard and CUDA drivers (177.84)
  • motherboard: ASUS P5N-T Deluxe (3 PCIe x16 slots)
  • 1x 8600GT (PCIe slot #2)
  • 2x 8800GT (PCIe slot #1,#3)

is anyone familiar with this kind of problem and is there a solution?

any help would be appreciated, thanks

People have reported automatic downclocking in the forums when the card was not getting enough power. This was almost always due to forgetting to plug in one of the PCI-Express power connectors on the backside of the card.

However, the 8800 GT does not appear to have any PCI-e power connectors, instead drawing all of its power from the motherboard. Can you try booting with just one 8800 GT installed and see what clock rate it shows?

thanks for the advice, i’ll try this and check the power connectors tomorrow - because today i’m not at work anymore

as for the PSU, its 850W

You might also check if the motherboard can power two 8800s only through PCIe. That can be quite a few Watts going through it even though G92 is more efficient than G80. A standard PCI-e slot can output about 75W, the 8800GT can reportedly require up to 135W (peak). PCI-e 2.0 can output 150W.

The new 8800GTS differs from the GT only by having one more MP (8 vs 7) and slightly higher clocks yet it has dual slot cooling and a 6-pin power connector. I have the impression that GT can have serious power issues on PCI-e 1.1 boards and possibly even on some 2.0 boards.

ASUS P5N-T Deluxe has only two 2.0 slots and one 1.1 (all x16). Did you make sure the 8600 sits in the 1.1 and 8800s in 2.0?

The 8800 GT requires a 6-pin power connector ( it is on the side, easy to miss, but it is there).

I think the 8800GT also beeps very loudly at you if you don’t plug in the power connector.

Ah, right. Thanks for the correction! (The 8800 GT cards I use are now 2000 miles away, and none of the online photos I looked at showed the edge of the card…)

Today we checked the power connectors and found out that this was not the problem.

Then we removed the 8600GT and reinstalled the drivers with the two 8800GTs plugged in - it was ok, untill we installed some tuning software after which the same problem occured again: one of the GPUs downcloaked on the same values as before (399/399/810MHz), and neither nVidia tuning software nor GPU-Z could determine the frequency of that GPU (so we had to use CUDA device query call to check the clocks).

Then we reinstalled the drivers once again, and it seemed OK. After that we reconnected the 8600GT and set it as primary display adapter and the clocks were still normal but the nVidia tuning software doesn’t seem to work correctly with multiple different GPUs: when we tried to change the profile from AUTO to “user-defined” the GPUs crashed (the software doesn’t support overclocking of each GPU separately). The clocks get all messed up when you try to make a change in the “user-defined” profile, even changing the fan speeds there crashes the GPUs.

The frequencies are normal (for) now, but we cannot overclock any of the GPUs and the only way to increase the fan speeds (which is 30% by default) is from the NVIDIA Control Pannel -> Performance -> Device Settings -> Create Profiles window (no idea why it works only from here and not also from -> Stored Profiles window).

So we think it is a purely software problem: the graphic drivers and NVIDIA Control Pannel make a system with multiple different GPUs unstable and/or lower it’s efficiency.

What are your thoughts on this?