Nsight cuda debugging pretty much unusable

Developer Tools Nsight Visual Studio Edition
1

Mostly debugging programs with CUDA + OpenGL interop:

  1. Randomly depending on edits to cuda kernels it suddenly becomes impossible to watch the values of variables
  2. Random GPU driver crashes under nsight but fine when run without

Really it feels like the state of GPU developer tools has taken a massive step back over the years, causing more time to be spent dealing with these kind of issues rather than getting actual work done.

There are all sort of insane hoops to jump through, previous gen cards needed for GL shader step through debugging, oh you want to step through debug dynamic parallelism with GL interop?, expansion of variables in watch windows if you can get them to work is insane for all but simple types etc.

Nsight 5.6
Win7 64bit
GTX 1080Ti + GTX 1080
VS2017 15.7.3
GeForce Experience Driver 398.11

2

I guess 1 and 2 are the same issue, you cannot see the var when driver is reset, I guess you shall try to disable the TDR.

3

I’ve tried longer TDR and it doesn’t seem to help unfortunately. Some kernels in the same executable run I can step through, others I cannot. Not sure at all at the moment what the differences are that cause this to happen.

4

I still think that TDR restarted your driver when you were doing cuda debugging, I guess 1080Ti is your display card, can you launch the monitor with env var CUDA_VISIBLE_DEVICES=1 then try to debug it again?