I’m having an issue where a CUDA function is storing an INF value in the very first variable of my array, but I don’t know why. I only found out by going through all my arrays one by one and printing non-zero values and happened to see the INF. The code provides no error and continues through. Obviously, this one INF propogates and ruins all subsequent calculations. I would like to know how to get notified when such a value is generated. Maybe that way I can figure out what is causing the odd value. In my minimal working examples with smaller matrices, this doesn’t happen.
There isn’t any automatic method to trigger a warning when an INF value is loaded or stored in GPU code, or is the result of a calculation. You can “manually instrument” the code with the isinf() type functions provided by the CUDA Math libraries.
Here is an example discussion showing its use. There are different versions for float vs. double. With a bit of searching you can find other forum discussions like that one.
1 Like
One possibility to automate would be to use custom C++ types, which just store a normal type, but where you overload all arithmetic operations to contain a check.