I have two tesla C870 cards and a 8500 GT in the same computer. I run fedora 7 in a Dell Precision T7400 2* quad Xeon CPU, and driver 169.09 in linux x86_64.
The bandwidth test gives the following results for a tesla card:
The other Tesla also gives a device to device of 9000 MB/s, and the 8500GT around 6000 MB/s. Why is the Tesla so slow ? Is there a way to check the memory clock rate?
Oops , there is a power cable missing, this must be the cause.
Did that fix the problem?
I have been facing similar bandwidth problems…
Yes, it fixed the problem.
Where did you find enough power cables for 3 graphics cards in a Dell T7400? I only found two 6-pin PCI-e power connectors in my T7400. I’m trying to figure out how to install a GTX280, which requires one 6-pin and one 8-pin power connector. Dell support didn’t have any ideas when I called them.
I’m in the same situation as you: a Dell T7400 with no 8 pin PCI-e power connector. The power supply is rated at 1000 watts maximum (!), but I couldn’t find any spec’s beyond that.
NVidia has a FAQ which states that in certain cases a 6 pin to 8 pin PCE-e adapter can be used successfully. The result depends on the power supply – the 6 pin connector should be able to carry 150 watts. The FAQ further states that one should not use adapters which convert two 6 pin connectors into a single 8 pin connector.
If you figure out how to power the GTX 280, let us know.
If you don’t mind, please tell us more about your setup. How do you power the two Tesla cards? Also, the two PCI-e slots in the T7400 are very close together, and it appears that one card would block the other’s fan. How does the air flow? Do you find that cooling is not an issue?
Thanks.
The specs for the Dell T7400 power supply are printed on the side of the power supply:
+3.3V @ 30A +5V @ 16A – maximum 150W combined
+12V @ 18A times four – maximum 950W combined
-12V @ 0.2A
+5Vfp @ 3A
1000W total
It’s not a standard ATX power supply, so I can’t swap it out easily. It uses a 24 pin and a 20 pin connector to the motherboard. Dell T7400 power supply connectors
P24 looks like an 8-pin PCIe power connector, but it my box it’s actually a 10-pin connector.
Each Tesla card is powered by two 6-pin connectors, so 4 are needed. As there wasnt that many in the power supply, I used the Y- cable provided with the Tesla that allow to have additional 6-pin connectors from unused 4-pin drive power connectors, and it works well.
The 8500 GT used for display doesnt require additional power connector, so no problem here.
I didnt experience any particular problem of overheating, although one intake is blocked.
I also intend to buy a GTX 280, but have no idea how to power it.
To come back on the topic of crappy bandwidth… Isn’t it possible that because of the use of 3 PCIe slots they downgrade to a PCIe8x instead of 16x?
because the normal memory bandwidth HtoD and DtoH should be around 4GB/sec I think?
Thanks,
Jordy
[edit]
bla bla bla… I read about the powercable :D nvm
[/edit]
I ended up using a 6 to 8 pin PCIe power adapter.
I measured the AC power draw of the Dell T7400 using a KillAWatt power meter.
box idle, no GPU = 185W
+ 1 CPU active +48W
+ 7 more CPUs +60W
C870 +80W idle / +95W running
GTX280 +48W idle / +95W running / +120W peak
Peak power draw with the GTX280 and 1 CPU active was 353W.
The alternative was to buy one of the dedicated VGA power supplies that fit in a 5.25" drive bay. These provide just +12V with two 6-pin and two 8-pin PCIe power connectors:
Thermaltake W0157RU 450W $70
ePower EP-350CD 350W $90
FSP Group BoosterX 3 400W $90 (only has four 6-pin connectors)
Just to get back on the bandwidth topic… I have both a GTX280 and an NVS290 in the T7400’s two PCIe x16 v2.0 slots. BandwidthTest reports:
NVS290 GTX280
pageable h->d 2.0 2.1 GB/s
h<-d 1.7 1.7
pinned h->d 2.6 5.7
h<-d 2.5 6.0
d->d 4.1 115.7
Huh, I just got a T7400 but I didn’t think it had a 8-pin PCIe connector, but looking at that Newegg link and the small 2-pin connectors… I know my T7400 has at least one very similar 2 pin connection, so maybe it does actually have an 8-pin.
I’ll check on that when I’ve got a minute and let you guys know.
I checked–right wires, wrong size. doh.