While debugging my CUDA application, it will sometimes crash for various reasons.
When I come to run the application again it can run notably slower (usually ~20 fps, now closer to 1 fps), I’m assuming this is due to the GPUs resources not being deallocated correctly.
The only way so far I’ve discovered to fix this is to restart my computer (using the same GPU for graphics and CUDA) however that’s rather inconvenient if this is happening alot.
I was wondering if anyone knows of a program, command or CUDA init flag I can use to clear/reset the GPU, I’ve looked around google and haven’t turned up much.
I got the same problem with a titan card. had to restart for gpu to free resources. latest cuda 5.0 here with latest gfx 320 driver released a few days ago.
The issue is that the card (for me) runs at about half clocks after a crash - it isn’t to do with deallocation, it seems to be a driver thing
A while back I wrote a workaround - if you let an opencl kernel timeout, the timeout resets the driver and the clock issue is fixed. It uses some code from the Nvidia SDK and some borrowed code off various tutorials
ahh, thats whats happening… it seemed to me that allocated resources was not freed. but yes, its running half speed, even if I see clock settings are correct.