I have a couple of questions hopefully you guys can give me a hand with.
I run Folding At home (protein folding simulations)
I just got a GTX 470 which is a monster of a number cruncher. I had no rig that wasnt in use so I mixed the GTX 470 in between 2 GTX 295’s on my 780i mobo. So I am running 5 instances of simulations running at once. I had to add force flags to all the 295’s to get them running with the GTX 470.
Now here comes my issue. Before I inserted the GTX 470 My GTX 295’s would run at 99% usage on each GPU. but after the instalation now all I can get out of them is 89%, while the GTX 470 runs at 99%
I am not certain why its behaving like this, or if it can be corrected any way except for removing the GTX 470. If it was only a single GPU I wouldnt worry about it, but with all 4 of the GTX 295s cores doing this it means I am loosing 40% a day.
Can anyone maybe enlighten me as to why this is happening? Or even recommend a possible solution for using these GPUs together without any type of conflict?
Do you know to what extent folding@home depends on PCI-Express bandwidth? If you have added a 3rd card, it is quite possible your motherboard (I haven’t looked at the 780i specs in detail) had to split one of your x16 slots into two x8 slots, cutting the bandwidth to one of your GTX 295 cards in half.
In addition, the front-side bus probably can’t saturate the PCI-Express bus to all cards at once, and it starting to bog down a bit more with the 3rd card adding to the load.
I don’t know the answer but do have some questions which may help diagnose things.
How do you measure “runs at 89%”? Is it accurate or some phantom loss? Looking at actual FAH units done over time is a real measure.
Does the 295 run at 100% when the 470 is installed but is not running? This will tell you if there’s a system PCIe issue.
Did you have to move the 295 to a new slot when you installed the 470? If so, is the 295 in an x16 bandwidth PCIe slot? (the bandwidth test SDK app can tell you if you’re getting full bandwidth.) This will tell you if there’s a PCIe issue with the 295 specifically.
If you run one instance on one 295 (one GPU) does it perform at 100%? Or do you need to run two instances (one on each of one 295 GPU card) or four instances (one on each 295 GPU, on each card)?
What’s your CPU usage? Pegged at 100% on all cores?
The EVGA 780 is true 16 X and all three operate at that speed when they are populated. Tecnicaly each GTX 295 is 2 GPUs, so with 2 GTX 295 and a GTX 470 thta brings the count to 5 GPUs. I wonder if somehow I have exceeded the bandwidth.
EG
I determine the percentage of GPU usage via GPUz sensor tab or with EVGAs precision tuner. I dint move any of the GPUs around. My PCIE slots are labeled from top to bottom as 1, 3, 2. The GTX 295s are in slots 1 and 2 with the GTX 470 in the middle between the two GTX 295s.
With this setup i am running 5 instances of the protein folding simulation, and the CPU usage is fluctuating between15 to 18% load.
I do not know what the limitations of F@H are with PCIE speeds. I had a K9A2 mobo that was 4 PCIE slots all at 8X and it was in fact much slower.
I pulled up the manual since I’m not aware of any consumer board that can do three PCI-E x16 2.0 slots at full speed. They are very clever in the marketing. One of the slots (I assume the middle one), runs x16, but PCI-Express 1.0, while the other run at 2.0. That’s half the PCI-E 2.0 x16 bandwidth on the middle slot.
However, from the description of your setup, that means that the GTX 470 is getting the half-bandwidth slot. So if you see the GTX 295s slowing down, my next theory is that you are saturating the front-side bus. which has to pass the DMA traffic to all of these devices. Even on fancy Core i7 systems, all it takes is two devices running full speed simultaneously to completely saturate the QPI. (Fortunately, transfers rarely run exactly simultaneously, but still…)
EG firsly I want to thx you for the help you given to the world to help out the people trying to help solved the problems of health care for certain illness’s…It shows big time you are not a selfesh person…Now to help with your problem with your Video cards. Seem’s to me the bandwidth, when useing the GTX470 cut into the 4 cpu’s of the GTX 295’s…so if you figure ( add up the crunching power ) 99% for the two 295’s 99%=198% now useing two GTX 295 at 89% X 2 = 178, now add the solo 470, at 99% =277 %, so you are actualy gaining appox 78% more crunching power useing the GTX470 alone with the 295’s. Hope you understood my meaning here, if not I’m sure some one will come alone and ref-phrase it into some meaningful #'s ;)…CHEERS
No I’m talking over all score with the 470 he gains crunching power, if he only had the 295, he would be running at 2X 99% =198 so it is better to have the 295 at a lower power to get the extra crunch from the 470…Some time you lose on one end to get a higher total output. Yes the 295’s are loseing 11% but the 470 is over taken the 22% loss and given him alot more back ;)…CHEERS
No I’m talking over all score with the 470 he gains crunching power, if he only had the 295, he would be running at 2X 99% =198 so it is better to have the 295 at a lower power to get the extra crunch from the 470…Some time you lose on one end to get a higher total output. Yes the 295’s are loseing 11% but the 470 is over taken the 22% loss and given him alot more back ;)…CHEERS