What is the behavior of CUDNN_LOSS_NORMALIZATION_NONE?

Hi, I am try to use cudnnCTCLoss api, but a little bit confused of “cudnnLossNormalizationMode_t”. To my understanding, the only difference inside kernel is that, “CUDNN_LOSS_NORMALIZATION_NONE” do not lauch a softmax kernel, while “CUDNN_LOSS_NORMALIZATION_SOFTMAX” would lauch a softmax kernel.
But I found that with the same input, I just do softmax manually before feeding it into cudnn with “CUDNN_LOSS_NORMALIZATION_NONE”, and do not do softmax with “CUDNN_LOSS_NORMALIZATION_SOFTMAX”, the gradients returned in above two circumstances are different.
Is there anybody who can explain it to me?
Thanks very much!

Hi @gaochyz ,
Please allow us sometime to check on this.

Thanks!

Hi @gaochyz ,
Can you please verify if the cudnnCTCLossAlgo_t parameter passed into the CTC API function is the deterministic algo?

Thanks!