CUDA Toolkit 10.0 Nsight 6.0 breaking at phantom break points

Accelerated Computing CUDA CUDA Programming and Performance
#1

Using Visual Studio 2017 Version 15.6.7
CUDA Toolkit 10.0
Nsight Visual Studio Edition 6.0.0.18227

GPU Quadro P2000 in TCC mode.

When running CUDA Debugging (Next-Gen), sometimes the debugger will stop in a kernel at a position where there is no breakpoint. Of course this destroys any ability to debug (e.g. if a thread conditional breakpoint is set, the phantom breakpoint stops on any thread).

Even if I remove ALL breakpoints from the kernel, the debugger will still stop at the same phantom point.

The Breakpoints viewer does not show the phantom breakpoints.

I have tried reopening the project, clean and rebuild.

Anyone else seeing this, or have a way to clear out those phantom breakpoints?

#2

My experience has been that when the debugger stops at a place where no breakpoint exists, you typically have a problem with your code there, and most likely a memory error.

#3

I just experienced the same issue, and it was a memory error in my kernel.

Thanks @ailleurfwvat. Your comment helped me debug this faster.

But is this the intended behavior of Nsight?
In this kind of situation, a message from the debugger as to why it stopped would have been extremely useful.

My toolset, just in case it matters:

Visual Studio 2017 Version 15.8.9
CUDA Toolkit 10.0
Nsight Visual Studio Edition 6.0.0.18227

GPU GeForce 1050

#4

I do not know. My experience with nsight has been flaky at best. Sometimes breakpoints hit, sometimes they do not. Setting a conditional breakpoint while debugging crashes visual studio. Variables passed by reference have garbage value when you hover them. Hitting F10 to move to the next line will sometimes work, and sometimes get you out of the kernel altogether.

It’s the only tool we have (that I know of), but in my experience it’s not great.

#5

Thanks for this comment. I started with CUDA only recently, and it helps to at least know that this is the way things are with Nsight at the moment.