My application is written in Matlab using parfor parallelization. This means that each worker is in a separate process. Each process has its own CUDA context. Because I have more processes than GPUs, my function often needs to wait for a GPU when it wants to do some work.
For better overall throughput, I want to perform my time consuming computation on the CPU if the GPU is busy working on behalf of another process. In my mex function, I want to implement logic like the following:
if( gpuIsBusy() ) computeOnCpu() else computeOnGpu()
But I don’t know how to write the gpuIsBusy function. This might be easy if everything were in a single process, I could use a critical section. To communicate between processes, I suppose I could use a named semaphore, but I’m hoping there’s a cuda call to make it easier.