Originally published at: https://developer.nvidia.com/blog/constant-time-launch-for-straight-line-graphs-and-other-performance-enhancements/
CUDA Graphs are a way to define and batch GPU operations as a graph rather than a sequence of stream launches. A CUDA Graph groups a set of CUDA kernels and other CUDA operations together and executes them with a specified dependency tree. It speeds up the workflow by combining the driver activities associated with…
This would be more impressive if there were any public details on the internals of CUDA Graphs. Based on what’s available, one would never expect an O(n) overhead after the initial graph upload.