How to use graph API for cuDNN to do layer normalization?

AI & Data Science Deep Learning (Training & Inference) cuDNN
1

I am trying to figure out how to use the graph API for cuDNN to do layer normalization.

I have written the enclosed test stub. I run it with CUDNN_LOGLEVEL_DBG 2 and get:

before fprop
after fprop
before op_graph

W! CuDNN (v90700 66) function cudnnBackendFinalize() called:
w! Warning: CUDNN_STATUS_NOT_SUPPORTED_GRAPH_PATTERN; Reason: !patternMatchOptionalLinearDAG(this->getEntranceNodes().front(), userGraph->getEntranceNodes().front())
w! Warning: CUDNN_STATUS_NOT_SUPPORTED_GRAPH_PATTERN; Reason: referenceGraph->patternMatchLinearDAG(userGraph)
w! Warning: CUDNN_STATUS_NOT_SUPPORTED_GRAPH_PATTERN; Reason: LinearPatternMatcher::matchPattern(userGraph)
w! Warning: CUDNN_STATUS_NOT_SUPPORTED_GRAPH_PATTERN; Reason: matchPattern(opg.getGraph())
w! Time: 2025-02-09T15:26:25.740454 (0d+0h+0m+0s since start)
w! Process=18146; Thread=18146; GPU=NULL; Handle=NULL; StreamId=NULL.

after op_graph
before engine
after engine
before engcfg

E! CuDNN (v90700 66) function cudnnBackendFinalize() called:
e! Error: CUDNN_STATUS_EXECUTION_FAILED; Reason: fillup_num_execution_groups()
e! Error: CUDNN_STATUS_EXECUTION_FAILED; Reason: status
e! Error: CUDNN_STATUS_EXECUTION_FAILED; Reason: ptr->isSupported()
e! Error: CUDNN_STATUS_EXECUTION_FAILED; Reason: finalize_internal()
e! Error: CUDNN_STATUS_EXECUTION_FAILED; Reason: ptrDesc->finalize()
e! Time: 2025-02-09T15:26:25.740620 (0d+0h+0m+0s since start)
e! Process=18146; Thread=18146; GPU=NULL; Handle=NULL; StreamId=NULL.

Failed: cuDNN error graph-api-example.c:267 ‘CUDNN_STATUS_EXECUTION_FAILED’

What am I doing wrong? Why doesn’t it match a pattern?

Thanks,
Jeff (http: //engineering.purdue.edu/~qobi)

graph-api-layer-norm.txt (17.4 KB)

4

Hi @qobi ,
Can you pls check on the following pointers and let me know if this works?

  1. Verify Graph Patterns:
  • Double-check that your defined operation graph matches one of the supported patterns in cuDNN and that the tensor shapes and data types involved are compatible.
  1. Check for Compatibility Issues:
  • Verify that your CUDA drivers and runtime are compatible with your version of cuDNN.
  • Ensure tensor layouts and data types align with cuDNN’s requirements for the operations you intend to perform.
  1. Debug Execution Failures:
  • Investigate any possible memory allocation problems, stream mismatches, or inconsistent tensor shapes while executing your operation graph.
  • Take advantage of cuDNN’s error reporting to glean more detailed insight into the execution failure’s underlying causes.
  1. Update to Latest Version:
  • If possible, update to the latest version of cuDNN to take advantage of bug fixes or enhanced support for specific operations or graph patterns.