I’m sure we have all been in the situation of hardware/software incompatibility where the products in question are made by 3 or 4 different manufacturers. They quickly form a circular firing squad and resolution is protracted.
I would like to put together a system and don’t really want to order $3000+ min hardware and find out it doesn’t work together. Happily almost everything is made by one company: nVidia!
Will the following work together:
Three nVidia 9800GX2 GPU cards,
One nVidia 780i or 790i motherboard with Nvidia chipset, nVidia approved memory, quad CPU, latest nVidia BIOS and drivers,
One 1200 watt Nvidia approved power supply, adequate cooling system
and, of course, Nvidia CUDA 1.0 or 2.0 ?
Second question:
What is the minimum PCIe requirement for nVidia 8000, 9000 and Tesla series cards for nongraphic use with CUDA. I know they must be x16 physical connectors. What about electrical channels? 16 and 8 work, what about 4 or 1? PCIe is supposed to auto negotiate the channels. Will your boards do the same?
I know the host/device bandwidth will go south but in some applications it is a negligible consideration.
I have posted this here rather than a email to nVidia as the answers may be of interest to other users of this forum.
Thanks, Skippy
PS I have already searched the nVidia website with no success.
It seems that TWO 9800GX2 or THREE 9800GTX was supported at most now.
I recommend nForce 790i Ultra SLI motherboard.
2)
If your cards are better than 8600GTS,I think PCI Express 16X is needed.
There is a bit of folklore going around that for CUDA you must have at least one CPU core per GPU. There have been conflicting anecdotal evidence in these forums on this point. I would like to get an official response on six GPU (three 9800GX2) and four CPU cores. Will it work?
8800GTX (for one) will function in an electrical x8, physical x16 slot.
I have a 790i and have plugged various combinations of the following in at the same time. A 8800GTX(OC), a Tesla, an 8800GT and two 8800GTS(512M). They will work for a few minutes. However, if you are planning on performance computing the cooling is a major critical hurdle. I have a huge full tower case and multiple fans, but one problem is it heats up the room as well. I am currently looking at Liquid cooling solutions, which threaten to invalidate the warranties. The 9800GX2 vents inside the case as well as out the back. If you plan on operating the proposed system for long periods of computation you definately need to budget and extensive cooling system and have good cooling in the computer room. As far as seperate cpu’s for each card, I believe all it really takes is seperate threads per card.
dont get 3 9800gx2’s, get two of them OR 3 9800gtx’s.
this is a user to user forum, you wont get an official responce from here, you need to contact nvidia directly for that.
i can say that i think you wqill have issues with 3 gx2’s but im sure 2 would be fine. and when people say you should have one core per gpu, that is to achieve optimal performance, not to make it work or not.
Thanks for the info. EVGA (for one) will NOT invalidate warranty if use use aftermarket water blocks; they also have the best general warranty and customer support. BFI and EVGA both offer some GPU’s with factory installed water blocks; they are slightly more expensive than DIY buying seperate items. If you DIY, the best place for good information on water blocks and cooling systems is the forum at xtremesystems.org.
I’ll settle for a semi-official response. There are plenty of Nvidia people on these forums. Your response indicates the problem. You think I might have problems with 3 GX2’s, but you don’t really know. nVidia has all the requisite hardware laying around and could throw something together quickly. WHY should the do this? BECAUSE their published literature on THEIR products does not give sufficient information to determine their compatibility.
I’ll be happy with their answer even if it is NO. It will give me an excuse to blow my budget and get a Skulltrail with dual quad Xeons. That may or may not work, I’ll take my chances; but if I am going to buy an nVidia board with nVidia GPU’s &ct. I think I should be able to find out in advance.
see how they show triple cards in the gtx but only gx2, showing you cant run 3
main reason is probably because 6 gpu’s just doesnt scale, or the drivers cant handle more than 4.
best thing to do is get 2 gx2’s, thats 4 gpu’s and is pretty good :P
btw the skulltrail requires fully buffered ram which is expensive and isnt as fast as regular ddr2 ram as its server class ram, designed for reliability and fault protection.
These are for SLI application. CUDA doesn’t support SLI and the GX2’s have to have their internal SLI disabled to function with CUDA. Memory (host to device) bandwidth requirements depend on the particular application; in some cases it can be negligible. In applications which may take days, weeks or months to run slow reliable server memory can be better than ultrafast DDR3. This bandwidth depends on the memory and also the PCIe links (version 1 vs version 2 and x8 versus x16) and there are many different varieties available.
You will certainly need 1 thread per GPU, and if your application requires lots of short kernel calls, then you will still observe some slowdown if you don’t have as many CPU cores as GPUs. This is just because the CPU threads will not be able to service the GPU immediately if they aren’t active. Longer running kernels should not have as much of a problem.