cudaSetDevice() on Cards not hooked to a Display

Accelerated Computing CUDA CUDA Programming and Performance
1

Hey,

I have a machine with 5 Cuda devices

4 Tesla and one Quadro.

The Quadro is hooked to the Display.

Since the Quadro should not be used for the calculation i would like to exlude it from

the cudaSetDevice(). Else the user experience isnt that good while running the calculation.

I found a forum entry that stated one should use the following: (from Stackoverflow)

cudaDeviceProp deviceProp;

int dev;

if(cudaGetDeviceProperties(&deviceProp,dev)==cudaSuccess){

    if(deviceProp.kernelExectimeoutEnabled==0){

        cudaSetDevice(dev);

    }

}

I did some testing with it, but my calculation still runs on all 5 devices including the Quadro.

Is there a proper way to determine if a Cuda-Device is hooked to a display and to exlude it?

2

You could use nvidia-smi and disable the Quadro for CUDA jobs.

3

Is there a more general way to do this?

When executing the program should check what graphics card is hooked to a display and then disable it for CUDA.

But Thanks for the tip.

4 
int deviceCount,slot=0;

       int *devloc;

       cudaGetDeviceCount(&deviceCount);

       devloc=(int *)malloc(deviceCount*sizeof(int));

       devloc[0]=999;

       for (dev = 0; dev < deviceCount; ++dev)

        {

        cudaGetDeviceProperties(&deviceProp, dev);

        if(deviceProp.kernelExectimeoutEnabled==0)

          {

           devloc[slot]=dev;

           slot++;

          };

        }

       // Now use devloc to assign devices

       cudaSetDevice(devloc[myrank]);
5

So this line return 0 when a Display is connected to the card?

I implemented the code, but the Quadro is still in the List of Cards available for Cuda!

6

You can disable the watchdog timer. Maybe you’ve done it a long time ago and forgot about it?

7

Ok, i disabled it. But now with the code above no device is added to the list…

(+ This removes the ability to debug with Nsight!)

8

I have the same problem and I couldn’t find any solution. I did quite a bit of research and I couldn’t get anything to work. I was told to use deviceProp.kernelExectimeoutEnabled==0 as well but this condition is fullfiled by my 2 graphics cards. I’ve seen so many people looking for a way to detect if one given device is connected to a screen and no one seems to have a solution.

I’m not an expert in OpenGL at all, but could one imagine a test (through some kind of rendering command maybe) that would fail if a device is not connected to a screen?

NVIDIA> is there any plan to add this property (to be attached to a screen or not) to cudaDeviceProp in the near future? I don’t know if it’s technically possible but that would be a nice feature to add for sure.