SampleNMT computes incorrect attention vectors in TensorRT 5.0.2.6

AI & Deep Learning Libraries and SDKs TensorRT
#1

It seems that sampleNMT incorrectly computes the attention vectors from the context vectors. There should be a tanh non-linear activation layer after W[c:h], as shown in Equation 3 https://github.com/tensorflow/nmt#background-on-the-attention-mechanism. To fix the bug, you should add a tanh activation at the end of SLPAttention::addToModel.

#2

Thank you for the feedback. I’ll bring this to our engineering team’s attention.

#3

Hello,

our engineers have committed the fix (Add missing tanh to sampleNMT attention) and should be available in a future release.

thank you