How to inline printf calls?

Hello,
I’ve noticed the printf function is not inlined in the sass, and thus requires a local memory stack to pass the parameters. Would it be possible to inline this function call?

This is a bit problematic because it makes it harder to know if the stack frame requirements of a kernel come from local arrays that the compiler didn’t manage to inline or just a bunch of printf calls.

Alternatively, nvcc should more clearly specify (using --resource-usage) how much of the stack is used for function call parameters, register spills and local arrays.

I’m specifically interested in the printf function because I use it to report out of bounds reads or writes.

Please do not use the printf in production code, not even in conditional branches. It is only meant as debugging tool.

And there were several issues reported on the forum from compiling in printf.

Nvidia should probably be more clear about the intended usages of the official printf.

If you need to report in production code, store the data in some memory structure and process it on the host side for printing.

I assumed printf was already implemented that way, with all the processing in the cpu side, which is why I’m surprised it requires a function call.

Anyway, I implemented a basic error reporting scheme that does not rely on printf.
Here is the general idea, in case someone is interested:

I declare an extern constant struct holding pointers to a buffer and a counter.
That way, all the kernels have access to the struct, and it only needs to be initialized once.

This requires to enable separable compilation (set_target_properties(mytarget PROPERTIES CUDA_SEPARABLE_COMPILATION ON) with cmake), which I hope should not impact performance as long as all device functions are annotated with an inline directive.

If an out of bounds read / write is detected, I just increment the counter atomically to reserve some space in the buffer and store some information relevant to the error: line, column, file, accessed index and accessed array size.

The file, line and column number can be obtained in device code with __builtin_LINE(), __builtin_COLUMN(), __builtin_FILE().

Then, after every kernel I want to monitor, I just have to check if anything has been written in the buffer. This implies a gpu/cpu synchronization though which I’m okay with. All of this can be disabled at compile time with a define, once I’m sure everything is correct.

Note that the __builtin_FILE() macro returns a const char* pointing to device memory. I build a string on the cpu using the device pointer by copying one char at a time until a null character is found. I would be happy to know if such a functionality is already available in cuda.

Conclusion:

  • This error reporting scheme does not use any stack allocations, as everything is inlined.
  • Errors must be checked manually on the cpu side and this implies a synchronization.

I would also be interested in knowing if there is macro similar to __builtin_FILE() that could give me the kernel name at compile time. This would help to distinguish between several kernels defined in the same file, in case errors are not checked after each kernel call.

That is obviously not possible. Device-side printf() inserts items into a circular buffer, so it requires code and data storage for implementing that. The host-side code from the CUDA runtime then pulls the data out of the circular buffer, formats it, and sends it to system I/O.

In general, CUDA inlines function calls in device code aggressively, because that is usually profitable. But that does not mean that it does that for every single function; it only does so where it is deemed profitable (by various heuristics) in terms of a performance vs resources trade-off. For example, even parts of the code implementing simple floating-point math operations like divisions and square roots remain as function calls.

If you have evidence / data that inlining device-side printf() would generally be conducive to improved performance, filing a well-reasoned enhancement request with NVIDIA would be the right course of action. It is not uncommon for trade-offs driving design decisions to shift over time, and not all such decisions are re-visited on a regular basis in the absence of outside prompting.