One problem with calculation precision in CUDA function


I found one problem in my kernel function regarding calculation precision.

In a division calculation: 1023000 / 5000000, the result should be just 0.2046, but my kernel function get 0.2046000063419342041015625… Why is there a residual after 0.20460000?

This difference directly results in other wrong results in my following codes.

Can any one help me to correct it? Thanks


Now I see that the difference is not because of the calculation, even if I directly assign value 0.2046 to the result array. Then I copy the values from GPU to CPU, and print out shows 0.2046000063419342041015625

Why is that?

I think it is related to the way the numbers are stored, as exponential. This makes some numbers more difficult to express. This article gives a good explanation about numbers in general.

Also everything which is beyond the 16th decimal is pretty useless

Floating point details are tricky but…

A 32-bit floating point to bits calculator shows that “1023000f / 5000000f” is represented as: 0.2046000063419342

0.125 * 1.6368000507354736 = 0.2046000063419342

So the extra bits past …342 imply to me that you are converting, casting or doing something even more subtle since dividing the doubles “1023000.0 / 5000000.0” would’ve had a different result.

Best thing to do is read up on floating point. Bruce Dawson’s blog is pretty educational: