I have some trouble debugging some CUDA code using Nsight.
so I’ve tried to debug one of the CUDA sample projects (sortingNetworks) using Nsight as well.
I reach the desired breakpoint, but when I continue execution I have an exception thrown:
CUDA module loaded: 03045ec0 oddEvenMergeSort.cu
CUDA grid launch failed: CUcontext: 40233744 CUmodule: 50617792 Function: Z18bitonicSortShared1PjS_S_S
and the program halts.
under normal visual studio debugger the code executes well.
I did not tinker at all with the project’s/files’ properties.
some background:
Nsight version: 5.2.0.16321
CUDA driver: 378.49
CUDA toolkit: 8.0.44
Visual studio 2013, under update 5
just installed driver 376.33, got the same exception…
tried to revert to each version from current, down to 375.95, but got the same exception every time…
tried to install CUDA Toolkit 8.0.61, got the same exception…
I’m on Maxwell (980 Ti), and have the same issue. Also tried to revert driver versions and CUDA versions to no avail. The lack of “debugability” of this problem is very frustrating :(
My OS is windows 7.
I’ve tried debugging both with enabled memcheck and disabled, to similar effect.
I have a separate GTX-1080, tried to install it as well on a another machine with similar environment,
got the same result.
moreover, the second GTX-1080 I’ve just specified was installed onto a machine that
ran and debugged the sample with a previous generation TITAN-X successfully, and now fails to debug with the GTX-1080.
Is there a way to collect valuable error data (a log file, a dump file, whatever)
that will assist in detecting the error and fixing it in the upcoming drivers / cuda toolkit?
I get grid launch failure repeatedly, each time I try to debug using Nsight.
I ran my code, and some of the CUDA sample code, mostly sortingNetworks.
I am able to hit a breakpoint if I set it early enough, but when I continue to the next kernel I get grid launch failure exception
If I try to debug using Nsight without any breakpoints, I also get the grid launch failure (at the same point in the same kernel as thrown in (1)) and the program halts.
all samples and my code work well if not trying to debug the kernels using Nsight.
this was tested on two separate PCs with two different GTX-1080s, running windows 7 with VS2013-update 5.
different drivers were tested from 375.95 up to the recent one.
all of the above works and debugs well on TITAN-X (Maxwell).
Bad news, I cannot repo it on both win7 and win10 with the same configuration, I’ve also tried the 1070 and 1060 and they both work well, maybe I should involve the dev to see what they can find out.
BTY I just see the latest driver is 378.66, I think you should have a try, maybe Custom(Advanced) → “perform clean installation” can beat this issue.
If clean installation and 378.66 still cannot solve it, I will raise a bug to track it.
I have tried a clean installation for 375.95, 376.33, 378.49 in the past.
Now I’ve tried a clean installation for 378.66 (which is from yesterday) and got the same error.
Is there any additional information you require, regarding the GPU, PC or environment settings?
versions:
cuda toolkit 8.0.61
nsight version 5.2.0.16321, standard
visual studio version - 12.0 (vs2013 with update 5)
geforce experience 3.3.0.95
please note that I have LogMeIn installed, and under display adapters, I have a LogMeIn mirror driver
in addition to the GTX-1080.
btw, the GTX-1080 is the only gpu installed on the PC, and the monitors are connected to it.
please let me know if you require any more information.
Nope, the task has been assigned to the dev, could you please give me you email, I can add it into the bug, then you will be notified when it’s updated
Opps, I don’t have authorization to add your email into the bug system, so I will still track this topic.
Actually some other customers got the similar issue, fail to launch kernel while debugging, could you please have a check if the nsight memcheck is off, also disable the TDR in nsight monitor might help.