How to implement a complete LSTM network with cudnn?

AI & Data Science Deep Learning (Training & Inference) cuDNN
1

I am building a semi supervised learning system with GA + LSTM. I need to build a multi-layer LSTM network with cudnn, which contains several LSTM layers and a softmax output layer. I found an example of how to build an LSTM layer with cudnn:

  1. cuDNN-sample/RNN_example.cu at master · Hardware-Alchemy/cuDNN-sample · GitHub
  2. https://github.com/PacktPublishing/Learn-CUDA-Programming/blob/master/Chapter10/10_deep_learning/03_rnn/rnn.cpp
    But I don’t know how to connect LSTM layer and softmax layer. There should be a density layer in the middle. The output of LSTM layer is 3D, and the input of density layer is 2D. How to convert?
2

Hi,

You can flatten the LSTM output to 2D version to match the input size of density layer.

If the original output is 3-D, [N, H, W], then the 2-D version would be [N, H*W]
Where N is the batch size, and K is the product of the other dimensions.

Thanks

3

Thank you for your reply. If the LSTM layer is directly connected to the softmax layer, it’s the same transformation, right?

4

Hi,

It depend on the output dimension of LSTM layer and dimension of your desired target/model.

If the output dim of your LSTM layer doesn’t match the output dim you’re predicting, a linear layer can be used in between LSTM & softmax layer to bridge the difference.

In case where the output of LSTM layer is 3D, and the input of density layer is 2D. You can flatten the LSTM output to 2D version to match the input of density layer.

Thanks

5

If the output dim of my LSTM layer matchs the output dim I’m predicting, no linear layer can be used in between LSTM & softmax layer. In case, the input to softmax layer is 3-D,how to select cudnnSoftmaxMode_t? And is there anything else to deal with?

Thanks!

6

Hi,

Please refer to below link for sample and details regarding cudnnSoftmaxMode_t:
https://docs.nvidia.com/deeplearning/sdk/cudnn-archived/cudnn_765/cudnn-api/index.html#cudnnSoftmaxMode_t
https://github.com/Hardware-Alchemy/cuDNN-sample/blob/7af1795d4207cc0ab7782210fdc1c272f1bca455/cudnn_samples_v6/mnistCUDNN/mnistCUDNN.cpp#L649

Please let us know in case you are facing any issues.

Thanks