In RNN is vectorSize equal to inputSize or hiddenSize?

[Shuai_Guo]

In cudnnSetRNNDescriptor_v8() I need to set inputSize and hiddenSize for an RNN descriptor. And I need to set vectorSize in cudnnSetRNNDataDescriptor().

In my mind, for a simple ReLU RNN, which is

the inputSize is x.size(), hiddenSize is h.size(). So where is vectorSize? Is it the same with inputSize?

[edp]

Hey Bud, in a similar boat.
Documentation is quite vague and really only the pytorch team are doing anything public with the sdk.
I’m pretty sure you are going to be creating an input(x) and an output(y) descriptor.
For x, your vectorSize will be the same as inputSize.
For y, your vectorSize will be the same as hiddenSize.

[sstepniewski]

Hi Shuai, @edp is right. The cudnnRNNDataDescriptor_t type can be used to describe the RNN model input or output, potentially two different data shapes. We also planned to use cudnnRNNDataDescriptor_t in RNN encoder-decoder configurations with attention. However, the latter effort was superseded by the multi-head attention API that employed cudnnSeqDataDescriptor_t.

Think of cudnnRNNDataDescriptor_t as a collection of fixed length floating-point vectors arranged in a 2D grid timestep x batch. Two layouts are possible: CUDNN_RNN_DATA_LAYOUT_SEQ_MAJOR_UNPACKED and CUDNN_RNN_DATA_LAYOUT_BATCH_MAJOR_UNPACKED depending on the order of vectors in memory. We also support the very first cuDNN RNN API layout CUDNN_RNN_DATA_LAYOUT_SEQ_MAJOR_PACKED that is indispensable when dealing with variable length sequences.

The cudnnRNNDataDescriptor_t type was introduced because the SEQ_MAJOR_PACKED layout was not convenient for some cuDNN users.

vectorSize in cudnnSetRNNDataDescriptor() must match the RNN inputSize when the descriptor defines RNN input, for example, xDesc in cudnnRNNForward(). The same vectorSize argument must match the RNN model output length when it is passed as yDesc in cudnnRNNForward().

RNN forward and backward functions (cudnnRNNForward(), cudnnRNNBackwardData_v8(), and cudnnRNNBackwardWeights_v8()) verify that xDesc->vectorSize == rnnDesc->inputSize and yDesc->vectorSize == rnnDesc->outputSize().

The RNN model output size is not always hiddenSize. Use the following pseudo-code to determine the RNN output length:

int outputSize() {
    if (isProjectionUsed()) {
        return projSize * (dirMode == CUDNN_BIDIRECTIONAL ? 2 : 1);
    }
    return hiddenSize * (dirMode == CUDNN_BIDIRECTIONAL ? 2 : 1);
}

Thank you for pointing out an issue in the cuDNN documentation. We are going to clarify the vectorSize argument of cudnnSetRNNDataDescriptor() in the cuDNN API Reference.