It seems preposterous to you because you are comparing it to an ideal case that is not necessarily achievable.
The ideal case is certainly an application that does not need to do any “on-the-fly” allocations. When I am teaching CUDA, I always strongly encourage this, and you can find suggestions online to “reuse allocations” and “keep synchronizing ops out of the work issuance loop”. In this ideal case, the proposed behavior does seem obnoxious. But its irrelevant, because we have no need to use the pool allocator. Of course you should do this if you can.
The whole premise here is that I have decided I need to do allocations during work issuance. If we posit that, what are our options?
Well, in the “old days”, we would have used
cudaFree, I guess). And what does
cudaMalloc do? It does exactly that synchronization that is depicted in that blog picture (or, at least, it has the potential to do that, depending on where/when exactly you issue it).
Now in the modern days, we have stream oriented memory pools, as discussed in the blog. What happens when the pool is running low? Well, you could either let the pool allocator try to “scavenge” potentially usable space (using this method, which may or may not have a noticeable impact on your app), or else it can “go back to the well”, and carve out more memory space, which is just like doing a new
cudaMalloc operation under the hood. And we are back to a synchronizing operation.
So among those choices, I think the potential dependency choice actually looks elegant. It might be the least disruptive to your carefully crafted work issuance loop. After all, is concurrent kernels really that easy to witness, anyway? How bad is serialization? I don’t know. And you don’t either, until we talk specifics, rather than generalities.
And if you get to a specific case, and don’t like that idea/behavior, turn the behavior off, as indicated in the blog.
How is it bad or preposterous to give you choices?
I’m not likely to argue this any further. Its OK if we disagree. Its the nature of community.