Not sure somebody ever talked this before.
Actually my question is, is there anyway(or how) to detect a GPU device is in use?
like I have two video card, one is hooked with monitor, the other one is not. I want to use the idle one for CUDA computation so I can avoid the “time out” problem.
How to tell the device 0 or device 1 is idle with cuda?
There isn’t a way of detecting an “idle” GPU, but then again just because a GPU doesn’t have an attached display doesn’t automatically imply it is idle.
Having said that, cudaGetDeviceProperties returns a structure which includes a couple of useful fields which can help with this: kernelExecTimeoutEnabled will tell you whether the card has a watchdog timer or not, and computeMode will, on platforms where it is supported, allow you to check whether a card is compute prohibited or not. Used together, you can “steer” device selection to usually end up on the correct card.