How does --use_fast_math affect division precision in CUDA?

Accelerated Computing CUDA CUDA Programming and Performance
1

How does --use_fast_math affect the precision of division operations in CUDA?

I wrote a CUDA kernel where enabling --use_fast_math leads to results that are completely different from the results obtained without --use_fast_math.

I understand that --use_fast_math enables faster but less precise implementations of certain math operations, but I did not expect the numerical difference to be this large, especially for division.

Could someone explain:

  • What exactly changes for division (/) when --use_fast_math is enabled?
  • Whether division is replaced by reciprocal-based approximations (e.g., rcp) and how many bits of precision are guaranteed?
  • In what cases the error can become significant?

Any explanation or references would be greatly appreciated. Thank you!

2

The primary effects for --use_fast_math I can think of on division:

If the division is embedded in other math, or used consecutively, there may be compounding effects. For example, the flush-to-zero of a subnormal number could be catastrophic on a subsequent calculation.

--use_fast_math implies a set of compiler switches for nvcc:

--use_fast_math implies --ftz=true --prec-div=false --prec-sqrt=false --fmad=true .

You could experiment with each of these individually, or perhaps in groups, to see the effect on your code. For the two items I listed, --ftz=true affects/causes the first item, and --prec-div=false affects/causes the second item.