I’m sorry, I did overlook that sentence in your original post.
Actually, I was just told late last week that for Vista and Win7, in fact the TDR timeout applies to all WDDM devices, even if they don’t have a display attached. I realize that the CUDA Toolkit release notes indicated the opposite; this was an error. (We just corrected the release notes in this regard this weekend.)
So clearly you can’t enable the timeout because you need to debug, as you’ve said. (Even if you hypothetically enabled the timeout and set it to a very long timeout period, that still wouldn’t help, because if you’re debugging, you might need an arbitrarily long time, and if you’re stuck in an infinite loop, you want to kick out of it as quickly as possible.)
Instead of using TDR, then, why can’t you use a counter in your kernel that keeps track of how many data structure nodes you have traversed. Pick some threshold of iterations that is way beyond what should happen when everything is working, and if that number is reached, either exit completely or use a conditional breakpoint in the debugger to stop and see what’s going on. So it’s like a timeout, except then it only counts when the kernel is actually actively doing something.
Does that help?