API Reference :: NVIDIA Deep Learning cuDNN Documentation

AI & Data Science Deep Learning (Training & Inference) cuDNN
1

Hello all,

I am currently using tensorflow with LSTM right now, but i’m running into a limitation where tensorflow LSTM will only work with the tanh activation function while still utilizing CUDNN. I don’t understand why CUDA limits the activation function type for LSTM. In this link above the RNN mode can specify LSTM, but you can’t do LSTM with RELU. I would expect there to be an RNN_LSTM_RELU mode in addition to just tanh. Can anyone explain why LSTM relu isn’t supported in CUDNN and if this will ever be supported? Thanks

3

Hi @arusse02,
Can you please tell us the urgency of having that mode?
We know the current API doesn’t allow a lot of tweaks on top of the standard RNN cells.
However we have a plan of making things more flexible in future releases which will allow users to be able to tweak the RNN cells while not sacrificing performance.

Thanks!