I assumed printf was already implemented that way, with all the processing in the cpu side, which is why I’m surprised it requires a function call.
Anyway, I implemented a basic error reporting scheme that does not rely on printf.
Here is the general idea, in case someone is interested:
I declare an extern constant struct holding pointers to a buffer and a counter.
That way, all the kernels have access to the struct, and it only needs to be initialized once.
This requires to enable separable compilation (set_target_properties(mytarget PROPERTIES CUDA_SEPARABLE_COMPILATION ON) with cmake), which I hope should not impact performance as long as all device functions are annotated with an inline directive.
If an out of bounds read / write is detected, I just increment the counter atomically to reserve some space in the buffer and store some information relevant to the error: line, column, file, accessed index and accessed array size.
The file, line and column number can be obtained in device code with __builtin_LINE(), __builtin_COLUMN(), __builtin_FILE().
Then, after every kernel I want to monitor, I just have to check if anything has been written in the buffer. This implies a gpu/cpu synchronization though which I’m okay with. All of this can be disabled at compile time with a define, once I’m sure everything is correct.
Note that the __builtin_FILE() macro returns a const char* pointing to device memory. I build a string on the cpu using the device pointer by copying one char at a time until a null character is found. I would be happy to know if such a functionality is already available in cuda.
Conclusion:
- This error reporting scheme does not use any stack allocations, as everything is inlined.
- Errors must be checked manually on the cpu side and this implies a synchronization.
I would also be interested in knowing if there is macro similar to __builtin_FILE() that could give me the kernel name at compile time. This would help to distinguish between several kernels defined in the same file, in case errors are not checked after each kernel call.