cuCtxSynchronize fails with latest nvidia windows drivers (581.80)

Accelerated Computing CUDA CUDA NVCC Compiler
1

After users have updated to the latest nvidia drivers (currently 581.80) on windows (linux does not seem to have any issues as of right now), my software is crashing, and after debugging my code it turns out that its failing on the driver api cuCtxSynchronize. prior nvidia driver versions this was working just fine.

Here is a psuedocode sample of the code thats causing the issue:

for(int device_id = 0; device < device_count; ++device_id)
{
  CUdevice device;
  CUcontext context = nullptr;
  unsigned int flags = CU_CTX_SCHED_BLOCKING_SYNC;
  CUresult res;

  res = cuDeviceGet(&device, i); // res is CUDA_SUCCESS, device is value 0 (first device)

  res = cuCtxCreate_v2(&context, flags, device); // res is CUDA_SUCCESS and context is not nullptr

  res = cuCtxSetCurrent(context); // res is CUDA_SUCCESS

  res = cuCtxSynchronize(); // res is now CUDA_ERROR_CONTEXT_IS_DESTROYED

  // store device and context for later use, never gets here though because cuCtxSynchronize fails
}

if i comment out cuCtxSynchronize, my software is able to continue actually compiling and deploying cuda kernels, however at any point where i need to sync with the device with cuCtxSynchronize, i get context destroyed

what exactly changed in the driver api from version 577 to 581 on windows? and how do i fix this?

2

just tried with the hotfix patch 581.94 but still same issue unfortunately

it must be something i’m doing wrong since other cuda applications seem to be working, i just can’t figure out what though. i do not have any other threads running in parallel at this time that may be attempting to access the device causing some kind of race condition, no oc code either