Meanings of CUDNN_BATCHNORM_SPATIAL_PERSISTENT

This mode may use a scaled atomic integer reduction that is deterministic but imposes more restrictions on the input data range.
For finite but very large input values, the algorithm may encounter overflows more frequently due to a lower dynamic range.

These two sentence are difficult to understand. Can you explain it in detail?
What’s the meaning of a lower dynamic range?

1 Like