I am using the graph API for cuDNN 9 with CUDA 12.3.
I am trying to do layer normalization.
I check the return codes of all calls to cuDNN.
I successfully create tensor backend tensor descriptors for XDESC, SCALE_DESC, BIAD_DESC, EPSILON_DESC, and Y_DESC. I successfully create a backend operation norm forward descriptor with nMode CUDNN_LAYER_NORM and phase CUDNN_NORM_FWD_TRAINING. I then try to create a (CUDNN_BACKEND_OPERATIONGRAPH_DESCRIPTOR. I set CUDNN_ATTR_OPERATIONGRAPH_OPS to have one operation, the previously successfully created CUDNN_BACKEND_OPERATION_NORM_FORWARD_DESCRIPTOR. I set the CUDNN_ATTR_OPERATIONGRAPH_HANDLE to a handle successfully created with cudnnCreate(). I then attempt to finalize the CUDNN_BACKEND_OPERATIONGRAPH_DESCRIPTOR and get:
E! CuDNN (v90700 66) function cudnnBackendFinalize() called:
e! Error: CUDNN_STATUS_BAD_PARAM; Reason: tensor_->isFinalized() == false
e! Error: CUDNN_STATUS_BAD_PARAM; Reason: invVariancePort.init( FUSION_IO_TYPE_OUT, const_cast<cudnn::backend::Tensor *>(op->getInvVarianceDesc()), this)
e! Error: CUDNN_STATUS_BAD_PARAM; Reason: node_ptr->init(desc_ptr)
e! Error: CUDNN_STATUS_BAD_PARAM; Reason: createGraphNodecudnn::fusion::NormForwardNode(&(*iter), all_nodes.back())
e! Error: CUDNN_STATUS_BAD_PARAM; Reason: populateGraphNodesFromOps(outputGraph)
e! Error: CUDNN_STATUS_BAD_PARAM; Reason: initGraph(userGraph)
e! Error: CUDNN_STATUS_BAD_PARAM; Reason: this->initGraph()
e! Error: CUDNN_STATUS_BAD_PARAM; Reason: finalize_internal()
e! Error: CUDNN_STATUS_BAD_PARAM; Reason: ptrDesc->finalize()
e! Time: 2025-02-08T12:02:25.003662 (0d+0h+0m+1s since start)
e! Process=13947; Thread=13947; GPU=NULL; Handle=NULL; StreamId=NULL.
It appears that it is complaining about invVariance. But according to the docs, this is optional.
I know that one can speed up the reverse sweep by saving the mean and inverse variance computed during the forward sweep. But according to the docs, this is optional.
What am I doing wrong?
Thanks,
Jeff (http: //engineering.purdue.edu/~qobi)