according to my testing, if you add another cudaSetDevice(0);
after the cudaDeviceReset();
call, the problem goes away.
I’m not suggesting that should be necessary, or that use of cudaDeviceReset()
like this should be a problem, but evidently it is in this case.
You could file a bug if this is a matter of concern for you.
As a general rule, I advise folks that there is no need ever to use cudaDeviceReset()
and I personally never use it or recommend its use.
It is possible gain some additional info with some experimentation with cufft and compute-sanitizer
.
If you omit all cuda runtime API calls in the test case (e.g. cudaSetDevice()
, cudaDeviceReset()
, etc. then the cufft call still works (returns a zero status) but compute-sanitizer
reveals something curious: a call to cuCtxPopCurrent
fails with an CUDA_ERROR_INVALID_CONTEXT result. This sort of makes sense. Depending on the preamble here, no CUDA context has been established, and apparently CUFFT needs that for its work. But in this case, the error report from CUFFT is zero, so CUFFT apparently knows how to handle that case (presumably it causes the CUDA runtime to create a new default context for its use).
But when we put the cudaDeviceReset()
in there, after runtime context creation was triggered by the cudaSetDevice()
call, then compute-sanitizer
indicates CUDA_ERROR_CONTEXT_IS_DESTROYED on a call to cuMemGetInfo
. This, apparently, cufft does not know how to handle, or assumes is an indicator of a serious problem, and so it returns error code 5 from the cufft plan call (CUFFT_INTERNAL_ERROR).
Adding another cudaSetDevice()
call after the cudaDeviceReset()
seems to cause the runtime to create another new valid default context, to replace the one that was destroyed. Now, when cufft checks context availability, it is happy.