Problems between CUDA versions

Hi everyone,

It is the first time I am posting here in this forum. I have previously done a project on CUDA 3.2 with the Thrust library version 1.3.0 and now I am working with the same project but on CUDA 5.5 (the toolkit now has the Thrust library already with it, it is the version 1.7.0) and it is not working as it was supposed to. The computer I am using is the same. The project is an image processing project and the images that I had were good with high quality but now some artifacts appear. Can you point me in some directions? Do you have any clue why this is happening?

Thank you so much,

You’d have to post WAY more detail for someone to be able to help… a small reproduction case would work, and would help you to find out what the real issue is… it could just be as simple as some function prototype or subtlety changed between thrust 1.3 and 1.7 or even CUDA itself and that’s affecting it… but again, pretty much impossible to tell without you doing some debugging on your own.

The program is huge, so I think posting code would not be an option. The computer I am using is a Processor 0-3: Intel Xeon CPU E5530 @ 2.400 Hz with 11.6 GB Memory, with a GPU Quadro FX 3800, 192 CUDA cores, 1024 MB memory.

To do debug do you have any suggestion? The problem here is that everything depends on each other, so it is quite hard to isolate parts of the code :S Moreover, debugging CUDA code is really hard.

If you think that any other info. that I can post can make you have a better idea of what can possibly be please let me know.

Thanks again

Take a single known correct input/output pair from the configuration that worked and follow it through the steps in the new configuration… check that the variables/outputs along the step of the processing are the same… presumably at some point you’ll see that something doesn’t match up and can further isolate that specific case to see what the issue is.

The only change you made was compiling under CUDA 5.5 and Thurst version 1.7 versus CUDA 3.2 and Thrust 1.3 on the same machine? Does the old compiled version of the algorithm still work correctly?

If the algorithm uses double precision code, make sure that you are compiling against sm_13 architecture, which your Quadro FX 3800 GPU is capable of. Other than that, I have no ideas… make sure you’re using the same compilation settings you were in the past as well.

Is it possible to have two CUDA versions installed in the same computer? If so how can I choose in which one to run?

Yes to both. I also tried in another computer with the CUDA 5.5 and had the same problems.

Concerning the compilation setting I am using the same I was in the past. About the “compiling against sm_13 architecture”, how can I check that?

Thank you so much,

Yes, it’s possible. At least in Windows, the program will look for the corresponding CUDA DLL files for the version you compiled it for, either in the current directory, or the system path.

Bear in mind, earlier CUDA versions also had specific compiler they were supported under. So, for example, if you compiled the CUDA 3.0 code with Visual Studio 2008, there’s 2 different things that changed… the C compiler AND the CUDA toolkit… there might or might not be a way of making older CUDA versions work with newer compilers and vice-versa… so keep that in mind.

Regarding the last bit, if compiling with Visual Studio, and your code uses double precision, you need to pass something like -arch=sm_13 to the nvcc compiler. See for example:

If it’s not a double precision issue, you’ll have to proceed with a known input/output and trace the calls through the program to see what changed like I mentioned earlier.

Lastly, it might or might not be possible to compile the new code under CUDA 5.5 with the older 1.3 Thrust version… assuming something that changed in Thrust is the issue… I would try that last before you attempt to debug as above, just because it would seem unlikely that the API would change without getting warnings or errors at compile time.