device in use How to detect the device is in use

Dear there,

Not sure somebody ever talked this before.
Actually my question is, is there anyway(or how) to detect a GPU device is in use?
like I have two video card, one is hooked with monitor, the other one is not. I want to use the idle one for CUDA computation so I can avoid the “time out” problem.

How to tell the device 0 or device 1 is idle with cuda?

many many thanks


There isn’t a way of detecting an “idle” GPU, but then again just because a GPU doesn’t have an attached display doesn’t automatically imply it is idle.

Having said that, cudaGetDeviceProperties returns a structure which includes a couple of useful fields which can help with this: kernelExecTimeoutEnabled will tell you whether the card has a watchdog timer or not, and computeMode will, on platforms where it is supported, allow you to check whether a card is compute prohibited or not. Used together, you can “steer” device selection to usually end up on the correct card.

Thank you very much, I think it can solve my problem.

because I have one card without attacking any output device, I think “kernelExecTimeoutEnabled” will tells me.

Best Regards