Multiple GPUs: finding one that's not busy

My application doesn’t benefit significantly from using two GPUs. A better use of a multicore computer with two GPUs would be to run two processes simultaneously, with each process using its own GPU. The processes don’t communicate among themselves.

I know how to use cudaGetDeviceCount(), cudaGetDeviceProperties() and cudaSetDevice() to explicitly select one of the GPUs to use. But suppose process one is already running and using one of the GPUs. When process two starts, how can it tell which GPU is busy with process one and which GPU is available for its use? I’d rather not require the user to keep track of this and explicitly select one.

I thought about using cuMemGetInfo() and seeing which GPU has more memory available, but this technique could get confused if the two GPUs don’t have the same amount of memory.

I was hoping to find something useful in the context driver functions in the driver API, but can’t find anything.

On linux, GPUs in use are open with the “mem” file descriptor in lsof. I’ve used this to put together a python script that can be used to determine which GPU is free on my multi-GPU server. The script also performs GPU reservations so that launching 2 jobs at once doesn’t result in both jobs on the same GPU. I will be posting this script to the forums as soon as I have the time to make the UI a little nicer.

If you are on windows, I have no idea how you would do this.

We’ve been requesting CUDA API functions for this since CUDA 0.8, at least.

Unfortunately that applies to any GPU access, ie: X. From my understanding CUDA interfaces with the card through the driver then to an OS abstracted hardware control layer called the “Resource Manager”. Unfortunately CUDA doesn’t support many of the somewhat necessary multi-app resource management features the Resource Manager provides.

Well, I’m running it on a headless server without X so it works fine for me ;) I was pretty sure it worked on my desktop with X too, though… I’ll have to check that.