I have a system running a gtx580 and two tesla c2070s on windows 7 x64. Past versions of the code have been able to run fine with one instance on each of the teslas. However I have started getting a strange error.
The 1st instance will run fine.
The 2nd instance (does not matter which card is run first) will give a “all cuda-capable devices are busy or unavailable” for the first two cudamalloc commands, then run fine. If put in a check to keep trying the cudamalloc until it works, then it fails twice, and works the 3rd time and the rest of the code runs fine.
Both teslas return cudaComputeModeDefault when queried.
Any help would be appreciated.