How to get fastest free GPU cutGetMaxGflopsDeviceId always same device

Accelerated Computing CUDA CUDA Programming and Performance
1

I am trying to operate on a shared machine which has 8 GPU.
Can anyone think of a good way of getting the fastest GPU which is not in use?
I think cutGetMaxGflopsDeviceId() looks at the hardware plugged in not if it is in use.
Does CUDA have a next free device function?

Many thanks
Bill

2

I assume by “free” you mean a device that is not currently busy with compute tasks. You may want to take a look at the NVIDIA management library API, NVML, which provides a function nvmlDeviceGetUtilizationRates(), for example. The full documentation can be found here:

http://developer.download.nvidia.com/assets/cuda/files/CUDADownloads/NVML/nvml.pdf

3

Yes, CUDA has a way to automatically select free GPUs. It also prevents more than one program from using a GPU at once. To use it, enable the compute mode you want with nvidia-smi, then simply launch your cuda apps that don’t have any cudaSetDevice(). The first call to cuda will implicitly create a context and a free GPU.

4

Dear Dr. Anderson,
Thank you.
I did not realise cudaSetDevice() was optional.
I thought I had to use it before I started using any other part of CUDA.
In the present case the GPUs are identical, but in general,
if I do not use cudaSetDevice()
will CUDA give me the most powerful free GPU?

Thanks again
Bill (Dr, 55.75)