It seems there are space-time tradeoff between cudnnRNNForward* and cudnnRNNBackward*, so if I share workspace variable between them, cudnn sometimes gives CUDNN_STATUS_BAD_PARAM. The solution is get workspace variable on every time when it needed.
The problem is that, error message and documentation is not helpful to this. Just showing CUDNN_STATUS_BAD_PARAM doesn’t allow users to guess this kind of logic behind. Please provide more information of this.
See more detail in https://github.com/FluxML/Flux.jl/issues/854#issuecomment-537990652