Most certainly not. The difference provides all the crucial advantages of the fused operation. FMA contraction being enabled by default is the one exception to the general rule of avoiding non-bit-accurate re-association in floating-point expressions. This was a conscious design decision, because maximizing the use of FMAs is crucial to computational throughput on GPUs, and it also helps accuracy of floating-point computations on average.
Because this is a deviation from the general principle, a compiler switch -fmad=false was added to the CUDA compiler very early on to facilitate turning off FMA contraction.
not really for fast math operations since i figure that these potentially give different results on different architectures anyway
If by “fast math operations” you mean the MUFU instructions, best I can tell their behavior has not changed since they were first deployed in G80 twenty years ago. Which is a good thing, because a lot of code is likely implicitly dependent on their behavior, in that code has been tested only with the instructions as originally designed, and it is impossible to say how much of it would break if the operations would change behavior.
One could argue that the compiler should optionally be given more leeway in transforming code with MUFU operations, in particular by threading the switch -use_fast_math all the way down to ptxas and defining what precisely the flag would enable at that level, but that is not something that exists now, best I know.
There may be a limited amount of non-bit-accurate constant propagation through MUFU operations today (I haven’t checked; you could check whether that is the case). Independent of MUFU instructions, operations with explicit rounding modes are generally “protected” from non-bit-accurate transformations since it is assumed that there is something “special” going on when these are being used: even with -fmad=true, an FADD and a dependent FMUL with explicit rounding modes will not get contracted into an FMA.
When CUDA was first created, we favored robustness in floating-point computations, as the target was the scientific computation market. Exceptions for performance were made very consciously and only to the degree needed to be competitive with Cg, which applied all sorts of fast & furious code transformations. This was a concession necessary to get CUDA off the ground as a product, as use of Cg was the established way in which early general-purpose computing on GPUs occurred.
This attempt to match Cg’s performance was how device function intrinsics, FMA contraction by default, and the -use_fast_math switch originally came about (note that this switch is so ancient that it does not follow nvcc’s usual command-line flag spelling convention).