Hello,
I’m trying to get my hands on the CuDNN library and implement the different RNN layers trying to get the same results as Tensorflow.
I reached my goal for the TF case return_sequence = True equivalent to seq2seq.
However when I try to implement the layer in seq2one, the cudnnRNNForward() function returns a BAD_PARAM.
Here is a copy of the logs:
I! CuDNN (v8500) function cudnnRNNForward() called:
i! handle: type=cudnnHandle_t; streamId=0000000000000000 (defaultStream);
i! rnnDesc: type=cudnnRNNDescriptor_t:
i! algo: type=cudnnRNNAlgo_t; val=CUDNN_RNN_ALGO_STANDARD (0);
i! cellMode: type=cudnnRNNMode_t; val=CUDNN_LSTM (2);
i! biasMode: type=cudnnRNNBiasMode_t; val=CUDNN_RNN_SINGLE_REC_BIAS (3);
i! inputSize: type=int; val=2;
i! hiddenSize: type=int; val=4;
i! projSize: type=int; val=4;
i! numLayers: type=int; val=1;
i! dropoutDesc: type=cudnnDropoutDescriptor_t:
i! seed: type=unsigned long long; val=0;
i! dropout: type=float; val=0;
i! nstates: type=int; val=18432;
i! inputMode: type=cudnnRNNInputMode_t; val=CUDNN_LINEAR_INPUT (0);
i! bidirectional: type=cudnnDirectionMode_t; val=CUDNN_UNIDIRECTIONAL (0);
i! dataType: type=cudnnDataType_t; val=CUDNN_DATA_FLOAT (0);
i! mathPrec: type=cudnnDataType_t; val=CUDNN_DATA_FLOAT (0);
i! mathMode: type=cudnnMathType_t; val=CUDNN_DEFAULT_MATH (0);
i! auxFlags: type=unsigned; val=CUDNN_RNN_PADDED_IO_ENABLED (0x1);
i! plan: type=cudnnPersistentRNNPlan_t; val=0000000000000000;
i! fwdMode: type=int; val=1;
i! devSeqLengths: location=dev; addr=00000013084FB400;
i! xDesc: type=cudnnRNNDataDescriptor_t:
i! dataType: type=cudnnDataType_t; val=CUDNN_DATA_FLOAT (0);
i! dimA: type=int; val=[5,3,2];
i! seqLengthArray: type=int; val=[5,5,5];
i! layout: type=cudnnRNNDataLayout_t; val=CUDNN_RNN_DATA_LAYOUT_BATCH_MAJOR_UNPACKED (2);
i! paddingFill: type=CUDNN_DATA_FLOAT; val=0.000000;
i! x: location=dev; addr=0000001308422000;
i! yDesc: type=cudnnRNNDataDescriptor_t:
i! dataType: type=cudnnDataType_t; val=CUDNN_DATA_FLOAT (0);
i! dimA: type=int; val=[1,3,4];
i! seqLengthArray: type=int; val=[1,1,1];
i! layout: type=cudnnRNNDataLayout_t; val=CUDNN_RNN_DATA_LAYOUT_BATCH_MAJOR_UNPACKED (2);
i! paddingFill: type=CUDNN_DATA_FLOAT; val=0.000000;
i! y: location=dev; addr=0000001308422200;
i! hDesc: type=cudnnTensorDescriptor_t:
i! dataType: type=cudnnDataType_t; val=CUDNN_DATA_FLOAT (0);
i! nbDims: type=int; val=3;
i! dimA: type=int; val=[1,3,4];
i! strideA: type=int; val=[12,4,1];
i! hx: location=dev; addr=0000001308422800;
i! hy: location=dev; addr=0000001308422A00;
i! cDesc: type=cudnnTensorDescriptor_t:
i! dataType: type=cudnnDataType_t; val=CUDNN_DATA_FLOAT (0);
i! nbDims: type=int; val=3;
i! dimA: type=int; val=[1,3,4];
i! strideA: type=int; val=[12,4,1];
i! cx: location=dev; addr=0000001308422400;
i! cy: location=dev; addr=0000001308422600;
i! weightSpaceSize: type=unsigned long long; val=448;
i! weightSpace: location=dev; addr=00000013084FB200;
i! workSpaceSize: type=unsigned long long; val=8390848;
i! workSpace: location=dev; addr=0000001315C00000;
i! reserveSpaceSize: type=unsigned long long; val=1216;
i! reserveSpace: location=dev; addr=00000013084FAC00;
i! Time: 2022-09-23T09:50:23.984824 (0d+0h+0m+0s since start)
i! Process=1628; Thread=19948; GPU=0; Handle=0000026F93CFC010; StreamId=0000000000000000 (defaultStream).
E! CuDNN (v8500) function cudnnRNNForward() called:
e! Info: Traceback contains 1 message(s)
e! Error: CUDNN_STATUS_BAD_PARAM; Reason: (xDesc->dimA[0] != yDesc->dimA[0]) || (xDesc->dimA[1] != yDesc->dimA[1])
e! Time: 2022-09-23T09:50:23.999784 (0d+0h+0m+0s since start)
e! Process=1628; Thread=19948; GPU=NULL; Handle=NULL; StreamId=NULL.
Error occurred: 3
From what I understand, it is not possible to have a different batch_size or seqlenght between input “x” and output “y”.
Is this really the case or is the way I am doing it wrong?