ATI/AMD card along an Nvidia card How to setup an ATI card for display and Nvidia card for CUDA?

Hello, I am new to the forums, and relatively new to CUDA.

I think this is the right place to ask this.

I have searched this forum, and the net in general, but I couldn’t find clear and specific answers, so here I am.

I have a windows 7 64 bit system with a Radeon card (a 5870, as it might be relevant), and I want to install a Nvidia card (a gt 430) for CUDA programming, while keeping all displays attached the the Radeon card, that is leave the Nvidia card for CUDA programming only. Now I believe it is not as simple as just putting the card in, in fact I am not sure if it’s even possible.

So is it? If so, does anybody know of a safe procedure to do such a setup, including a right order of installing the hardware, the drivers… also, is there any kind of special driver one needs to install? I have read that to enable PhysX on a secondary Nvidia card hacks are needed for newer drivers, but that’s for PhysX.

If anybody has tried and succeeded with such a setup, I have a couple more concerns. Is it possible to have graphical output with such a setup? I mean, if the CUDA code involves graphics (CUDA OpenGL interoperability) with display data stored in the CUDA graphics card, will it be somehow possible to have its output out from the Radeon card? This might sound like asking too much, but I know this is possible with a two Nvidia card setup with no SLI (2 independent 460 cards), with one having all the displays and the other dedicated to CUDA (I really dont know how this happens, but I am sure this is the case, as CUDA debugging is made possible in this setting and the debugging does not allow one card to run both CUDA and display output, so one -unattached to output- must be doing the CUDA/graphics, and the other just outputting the other card’s results). The second, lesser concern, is performance hit in games, will such a setup actually affect the Radeon card’s performance?

Thanks in advance to all,

Gorune

About a year ago I tried to do this with a Radeon 5870 and an Nvidia GTX285 with Windows 7 64bit. I ran into an immense amount of problems. The order in which I tried the driver install was ATI first then Nvidia. The machine recognized that both cards were installed, but when I went into the Nvidia Control Panel->Nvidia System Configuration the specs were blank. I also found that the OpenCL drivers for ATI were conflicting with Nvidia’s OpenCL drivers. I then went ahead and tried to run some CUDA apps and it said no CUDA capable card was installed. I didn’t spend too much more time with it because I was selling the 5870 anyway because of the terrible OpenCL support and memory capping issues. The conclusion for me was on Windows 7, ATI and Nvidia do not play well together. If you want to use Linux then this is possible and has been done. Another thing to note that after this attempt my Windows 7 build was ruined (even with Driver Sweeper, Registry cleaners, etc). I would have constant graphics glitches and freezes. I had to reformat and reinstall Windows 7 to get everything stable again.

Thanks for the warning! I was literally about to try it out, prepared all the drivers, even dusted the pc from the inside! I guess the best thing would be to put the ATI card away, at least for the moment. If I do that, do I need to remove the ATI drivers before installing the Nvidia card? Sounds safer, but it might make trouble for the card and Windows.

I am not very familiar with Linux, but if you have some clean source that explains how things are done, then maybe I can give it a try. But then, there’s the IDE problem and maybe some cross platform code issues…

Also, you say this was a year ago, is it possible that the newer drivers are cleaner? Or is it purely because of how Windows 7 deals with the drivers?

Thanks again, I hope there’s a workaround to this, because, somehow, it really feels possible.

It’s absolutely possible - my system is a GTX 295 + HD 5770, but it’s a real pain to set up. Driver installation order matters, as does which PCIe slot the cards are in as well as which one drives the output. Basically, the ATI card had to be in the primary PCIe slot, and have ALL outputs plugged into it. You’ll also run into problems when you go to update drivers, since that’ll likely as not break the system forcing you to go through the installation from the beginning.

It does, however, work, with cuda running properly, and opencl detecting and working on all devices.

The system is Windows 7 64 bit.

Does Parallel Nsight debugging work?

I’m looking at a new system with Windows 7 64 bit, HD 6970 Primary card and GTX 570 second card.

I don’t know if the Nsight debugger works in such a setting, it would be really interesting if does though.

Before that however, we need to know how to get CUDA running first. Keldor314, you said you had CUDA running with proper installation, but you didn’t say what worked for you. From your post we understand that the ATI card goes in the primary PCIe slot, and the Nvidia in the secondary slot, with all outputs coming from the ATI, however it is not clear the order in which you installed the hardware and their drivers. This might be of some importance seeing the horrible results Mesa_Boogie271 had.

What could be the reason of Mesa_Boogie271’s setup failure? He (or she) said he installed the ATI drivers first, and then Nvidia. Now this seems a reasonable order (hardware installation order is missing though), what could have gone wrong? Should the Nvidia and CUDA drivers be installed first before ATI’s display drivers are done? Usually when a driver is installed the system requires a restart, could it be Mesa_Boogie271 installed everything then restarted?

Such combination and orders can go on for pages, so please tell us what you did (whether it failed or succeeded), in as much detail as possible, and we might be able to get a nice guide on how to setup such systems.

I had the ATI Radeon 5870 in the primary slot with all monitors plugged into it with the Nvidia GTX 285 plugged into the second PCIe slot. I first installed the ATI card and drivers and then installed the Nvidia GTX 285 card and installed the drivers. I didn’t feel like spending too much time on this because I was selling the ATI card anyway and had a second GTX 285 laying around ready to SLI and use as a compute card. My main interest in all of this was benchmarking ATI vs Nvidia in OpenCL. The strangest behavior for me was checking the Nvidia Control panel and having it recognize the GTX 285 but it didn’t recognize any of the System Specs (CUDA cores, Clock Speed, etc) for the card. This caused the problem of saying I didn’t have a CUDA capable card. You could possibly attribute some of this to the fact I bought my Radeon 5870 the first month it came out, and the drivers were still in there infancy. Hopefully things are more stable now on ATI’s end. I do want to repeat that this really is easier on the Linux side because of how you can specify driver locations.

After removing my Radeon 5870, unistalling drivers, running driver sweeper and installing the Nvidia card and drivers. I had constant graphics freezes and noticed ATI processes still running in the background. I tried to remove all these processes but never found the root cause. It was easier for me to reinstall Windows 7 then track that down to get everything stable with just the Nvidia card.

A completely other issue was trying to get OpenCL to work with my Intel i7 x86_64 chip and Nvidia GPU at the same time. The only drivers that exist for x86 are the AMD x86_64 ones. When installing the AMD x86 drivers they actually overwrote my Nvidia OpenCL drivers. This caused another day of headaches to get working. One would think it would be easy to get OpenCL working with CPU and GPU’s at the same time because you know that is the point of heterogeneous computing, but it is not. Just wanted to mention this point and to be very cautious about the paths you install everything when attempting this.

This is quite disturbing honestly, because that’s what I had in mind and it makes sense, as an order to do things. Let’s hope this was a driver issue, and that’s why could you please give us the driver version you were using, or if you don’t recall, the range of drivers that were available back then?

I wonder when Keldor314 build his working setup and what drivers he used.

This is really frustrating! I can’t afford reformatting the pc, I need a safe way to get CUDA working.

Intel have a beta OpenCL 1.1 that is not tied to a video card.

http://software.intel.com/en-us/articles/download-intel-opencl-sdk/

This site has some info on it

http://www.geeks3d.com/20110202/intel-opencl-1-1-sdk-updated-printf-in-opencl-c-kernels/

Wow nice finally! I have been twiddling my thumbs waiting years for Intel to release a driver for x86. I attended Super Computing 2009 conference and I even spoke with the Intel rep for OpenCL working with Khronos and he said they would have one out shortly. Looks like it only took a couple of years.

Would it be better/possible to multi-boot two different installations of Windows 7 64 bit?
i.e. one with AMD drivers the other with NVIDIA drivers. With both cards physically installed but without a driver for the other card.

Obviously this wouldn’t solve the two card debugging requirement of nsight debugging but at least you wouldn’t be juggling two drivers installed together.

That might be possible, but one would have to plug and unplug the display every time. If Keldor314 had it running then it should be fine.

I will go ahead and try it soon, probably over this weekend. I will be using the latest drivers from both companies, but the AMD driver without the OpenCL component.

The plan:

1- Update latest AMD driver for the video card,
2- Add the Nvidia card, and install its drivers and CUDA

It looks simple, and I am not looking forward to the “it’s a real pain to set up” part, which of course does not appear in the plan.

Wish me luck.

On the dual boot note, if you have an extra hard drive or know how to partition correctly then you could do a seperate “test” build of Windows 7 x64. With this build you could go ahead and do the current plan and if it happens to mess anything up then you still have your real Windows 7 build on a seperate drive or partition that you can always go back to. This would honestly be the best route to take if you can’t afford a reformat.

It’s also worth noting that I had it not running quite nicely as well, so to speak. Unfortunately, I don’t remember the actual order I did stuff in, only that it seemed to matter :-(

I’m going to attempt to update my drivers to the latest versions - I’ll let you know if I find out anything, assuming I don’t screw up everything, a distinct possibility.

Okay… I uninstalled the ATI driver, BANG, black screen. Force restart after a couple minutes. Driver is now correctly uninstalled and I have VGA compatible display.

Next thing, I uninstall all ATI software with the catalyst install manager in add/remove programs. Windows notices the Radeon, now with no driver, and “helpfully” reinstalls it. This may take a while :dry:

Hey Keldor314, thanks for your attempts, I hope things go well with you.

I will do my first attempt in a couple of hours, once moving files from my extra hard drive is done. It will not be a perfect test, as the windows I will be installing is the Windows 7 32bit and not the 64bit, had to download all drivers again… No worries though as I plan to get my hands on a 64 bit copy on Monday and try it out the same day. I will keep record of the steps, these will generally follow the 2 steps plan I suggested in a previous post. I will be using the AMD driver with no OpenCL component, just making sure there can be no conflicts on that front.

Ok, installing the Nvidia driver first seems to be the way to go. I have Cuda working (though no opengl or directx interop - not completely surprising since the display card is ATI), as well as openCL, though the ATI platform isn’t showing up properly. I’m going to try uninstalling and reinstalling the ATI driver next since the last time it reinstalled itself prematurely.

One very important thing is that in order to get OpenCL working after the update, you have to go into the registry and delete the references to the ATI and AMD opencl dlls from HKEY_LOCAL_MACHINE/SOFTWARE/Khronos/OpenCL/Vendors after you install the nvidia drivers but before you install the ATI ones.

Ok, time to report.

After cleaning the hard drive, I installed the 32 bit Windows 7.

Then I installed the latest AMD driver with the Catalyst Control Center, but no OpenCL. I also installed an antivirus.

I put the gt 430 in, which sat 2-3 mm from the Radeon, blocking 1/3 of the fan, as a result, temperature rose almost 10 degrees (celsius) from the previous setting, in idle.

I then tried to install the normal (not developer) driver, but it would stop at 70% of unzipping and crash, it must be some corruption while downloading, and to think it took me an entire day to get it.

So I moved directly to the developer driver, installed it with no problems (clean install), but without the 3D vision component. I tried to keep things to a minimum in general.

Installed the CUDA toolkit 3.2 32 bit version. Installed latest Nvidia GPU computing SDK 64 bit version (blame the internet, 300+ Mb for the 32 bit was too much, and I had the 64 bit version from before, as the plan was to stick with 64 bit). This was a major annoyance, because it meant I couldn’t try CUDA applications, without recompiling the codes for 32 bit systems, and I didn’t have a visual studio on the new Windows build. Luckily I found a CUDA executable on my memory stick! And it worked! CUDA simply worked!

Now this is the good news, the bad news, maybe some of you don’t even care and it was expected actually, is that the CUDA/OpenGL interoperability doesn’t work, that is applications which have data operated on the GPU that are directly displayed without being copied back to host memory and then displayed, do not work. I wasn’t really expecting the Nvidia GPU to send the data to the AMD GPU for the display, but it was worth a try (especially since it somehow possible with 2 Nvidia cards).

I will try to attach a display to the card, and see what happens.

On my main build, I have not installed any Nvidia drivers, and in fact simply disabled the device. Before disabling it however, I fired a game, and I had 1-2 frames per second, after disabling, it was back to fluid 60 fps (fraps). I hope I get no interference from it as it is now. What does disabling a device from the device manager actually do?

I will keep doing a few more tests in the coming days, and switch to a 64 bit Windows, and keep you guys updated.

Second batch of updates. (Summary below)

I successfully installed the Windows 7 64 bit. Then proceeded with the same order of drivers. The only difference is that the Nvidia card was already installed. So the order was, ATI driver with no OpenCL, Nvidia developer drivers, CUDA Toolkit v3.2 64 bit, Nvidia Computing SDK 64 bit. With both displayed attached to the ATI card, things worked just like they did on the 32 bit system, that is, computations worked, but OpenGL or DirectX/ CUDA interoperability programs did not work (as expected, again).

At this point I can say that all is well for anyone who wants to use an Nvidia card as a CUDA computational device, in fact graphics can also be done, if the data is copied to the CPU then used for display (I have tested this with one of my programs). The 2 problems are with the interoperability and GPU debugging. I have not tested the last claim, but I am pretty sure it is not possible, you do need 2 CUDA architecture cards to debug.

I decided to plug a monitor to the Nvidia card, so that each card has one display. Things started to look unstable, with ATI tray tools not running, a sluggish overall reaction of the system, and the constant nvXDsync.exe in the background, using an single core to the max and about 7Mb of memory. I couldn’t stop the process. It was not looking good with CUDA either. SDK examples would start but not run, I did not try all of them, but any CUDA part seemed to stop the program, without crashes, just stop.

I then changed the screen settings, so that the only available display was the Nvidia display. This instantly stopped the ATI card fan, or it dropped to a very low speed, and of course shut the corresponding screen. After a while nvXDsync.exe stopped, and things started to look much better, with basically everything running, CUDA non graphical, graphical, and interop graphical (tested many SDK examples).

To have a more or complete test, I set up CUDA on Visual Studio with syntax highlighting and intellisense, and tried the “mandelbrot” example, and everything seems to be working.

Finally, I re-enabled the ATI card (brought back the fan noise), nvXDsync.exe popped too bringing some sluggishness to the system, things stopped running again, also, oddly enough, ATI tray tools worked fine, I could control the fan speed. Disabling the Nvidia card at this point did not put the system to normal. This setting is quite frustrating, the mouse does not work on the desktop (either busy, or clicks don’t work).

Summing up, an order of hardware/driver installation that enables CUDA on an ATI/AMD and Nvidia hybrid machine is, ATI/AMD card installation, with its driver, then the Nvidia card, with the developer driver then CUDA toolkit and then the Nvidia computing SDK. With all (1 or more) displays attached to the ATI card CUDA works whenever data held by the Nvidia GPU is not meant to be directly output from the card itself. To have that, one has to attach at least one display to the Nvidia card, and disable the ATI one, essentially making the system Nvidia only (and not a hybrid), else the setup cripples the system, and CUDA does not work. No worries then if OpenGL,DirectC/CUDA interoperability is not of interest.

I hope this was or becomes of help to people trying this sort of thing.

  • NOTE: no VGA cards were harmed during the making of these experiments.

Gorune