I’m having a problem with my CUDA based command line application. The kernel is by far the most time-consuming portion of the application, accounting for ~97% of the applications run time. This is also a fairly long running application. Depending on the data on which the application is executing, it makes several thousand calls to the kernel. This whole process takes on the order of 1 to 2 minutes. On occasion, particularly during testing, its desirable to stop the application before it completes its computations. Normally, this would be done by clicking the ‘Close’ button on the command prompt window. However, it seems that if that is done while a kernel call is being executed, the whole computer locks up, then crashes, resulting in a restart. I’ve implemented a second thread which monitors user input, waiting for a ‘q’ character, which then safely stops the computations. However, I still catch myself going for the close button on occasion, and I can’t imagine users of the application will be any different.
First off, am I doing something wrong? Is this normally how CUDA works? Secondly, is there some way I can interrupt the kernel gracefully? Or perhaps some means of setting the ‘close’ button behavior?
Some relevant information:
OS: WIN 7 64 bit
CUDA version: 4.0
binary: 32bit windows command line
I’ve actually solved this on my own. For anyone who might be interested, the solution was essentially to reduce the number of threads launched for a given kernel instantiation. To make up for this, I launched the same kernel a number of times, with an offset parameter to account for the problems with unique thread identifiers.
The problem was that Windows 7 doesn’t give the program the ability to handle a close event like XP and prior versions. Instead, it gives the program a short period of time, and if it hasn’t reached a ‘clean’ stop state, it just kills the process in the middle of whatever it was doing. In a normal program, this doesn’t have much in the way of consequences outside of any data the program is operating on. With CUDA however, if a kernel is being executed and goes beyond this short period of time, serious problems surface. I’m guessing at this point, but I would imagine that any host-side processes are killed by Windows, but the kernel is still executing. Once it finishes, it no longer has any process to return to, as Windows has killed it. This probably results in a device error. In my case, and I imagine most people’s cases, the CUDA device is also the display adapter. Apparently this issue is significant enough to bring down the entire device, ultimately resulting in the whole computer becoming unresponsive.