Urgent. Need help MultiGPUs bug

I try to understand what i can do multiGPUs. I make a small program that perform 3D gradient computation on 2 Quadro 5600 GPU. I give them the same input, and hope that they will produce same output.

However when i run the code, the test failed most of the time. Can any one try and explain why that happen.

Thank you. I really need to understand why it failed, without it i can not continue my research. I’m stuck.
multiGPU.tar.gz (2 KB)

The first problem I see is:

cudaArray* d_array3D = NULL;

float* d_data;

float* d_grad;

are all global and therefore shared between threads. You need to have separate variables for each thread, otherwise one thread will overwrite the address placed there by another thread.

I think these variable will be created per context, but it is likely that I’m wrong.

Thank you, you save me.

By the way i don’t have separate variable for cudaArray but it still works. Do i really need separate variable for cudaArray

You definitely need them for pointers to global memory. You don’t need them for pointers to constant memory (Symbols). Not sure about arrays.

Do you know where in the Programming Guide i can find all these information

The programming guide states that constant variables have to be defined at file scope. The implications are obvious.

The programming guide isn’t going to tell you that you need separate CPU variables to avoid race conditions between threads. This is a basic multi-threading concept. (This sounds somewhat confusing, but remember the pointers to device memory actually exist in the CPU’s memory space.)

If it helps, I’ve learned you need separate variables for each thread/GPU/CUDA context for everything except texture references and constant symbols.