cudnnNormalizationForwardTraining() vs. cudnnBatchNormalizationForwardTraining

[wonseok.shin]

I’m wondering what are the differences between cudnnNormalizationForwardTraining() and cudnnBatchNormalizationForwardTraining().

Initially I thought cudnnNormalizationForwardTraining() would perform more general normalization than batch normalization, but its documentation seems to suggest that it performs just batch normalization. Also, the documentation indicates that both functions are based on the same paper entitled “Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift”.

What is the critical difference between the two functions?

[AakankshaS]

Hi @wonseok.shin

@wonseok.shin

What is the critical difference between the two functions?

I believe there is little to none. cudnnNormalizationForwardTraining() was intended to be a general API which could potentially cover different kinds of normalizations.