Dynamic Control Flow in CUDA Graphs with Conditional Nodes

Technical Blogs & Events Technical Blog
1

Originally published at: https://developer.nvidia.com/blog/dynamic-control-flow-in-cuda-graphs-with-conditional-nodes/

CUDA Graphs can provide a significant performance increase, as the driver is able to optimize execution using the complete description of tasks and dependencies. Graphs provide incredible benefits for static workflows where the overhead of graph creation can be amortized over many successive launches. However, nearly all problems involve some form of decision-making, which can…

3

Nice feature! Are there any plans to support nodes with different contexts? Also the post seems to focus on the runtime API but are the memops nodes from the driver API allowed?

4

Nice post!

I’ve successfully run conditional nodes, but the same fail for me if I instantiate them to be launched from device.

…


            //Add do-while node
            cudaGraphNode_t whileNode;
            cudaGraphConditionalHandleCreate(&handle, g, 1, cudaGraphCondAssignDefault);


            cudaGraphNodeParams cParams = {cudaGraphNodeTypeConditional, {0, 0, 0}, {0}, 0};
            cParams.conditional.handle = handle;
            cParams.conditional.type = cudaGraphCondTypeWhile;
            cParams.conditional.size = 1;
            GPU_ERR_RET(cudaGraphAddNode(&whileNode, g, &node, 1, &cParams));
...

cudaGraphInstantiate(&graph.exec, g, cudaGraphInstantiateFlagDeviceLaunch);

it always fail with
- Code = 1 (cudaErrorInvalidValue) -

Are conditional nodes supported for device launch?

Thanks!!
E.