Can ```cudaDeviceSynchronize()``` from two threads/processes cause conflict?

Accelerated Computing CUDA CUDA Programming and Performance
#1

We have an application where two processes are mapped to a same GPU. In each process, we call cudaDeviceSynchronize() to check CUDA error messages. We saw weird GPU error from running the two processes. I am suspecting if calling cudaDeviceSynchronize() simultaneously caused the GPU error. Maybe calling cudaStreamSynchronize() is a better idea? Can someone confirm my guess?

I noticed this answer. From this, I understand that “device” is also a logic concept per process if MPS is not enabled. Then what if MPS is enabled?

#2

There is nothing about the idea of cudaDeviceSynchronize() being called simultaneously, with no other sources of error, that should cause spurious errors. With or without MPS.

If your question is actually, “can an error in one process cause an error in another process”, then I would say without MPS generally no, but there may be corner cases.

With MPS, see here.