Hello,
I found one problem in my kernel function regarding calculation precision.
In a division calculation: 1023000 / 5000000, the result should be just 0.2046, but my kernel function get 0.2046000063419342041015625… Why is there a residual after 0.20460000?
This difference directly results in other wrong results in my following codes.
Can any one help me to correct it? Thanks
Hi,
Now I see that the difference is not because of the calculation, even if I directly assign value 0.2046 to the result array. Then I copy the values from GPU to CPU, and print out shows 0.2046000063419342041015625
Why is that?
I think it is related to the way the numbers are stored, as exponential. This makes some numbers more difficult to express. This article gives a good explanation about numbers in general.
https://developer.nvidia.com/content/everything-you-ever-wanted-know-about-floating-point-were-afraid-ask?display[%24ne]=defaultnppcuda-downloadsuser%2Floginsites%2Fdefault%2Ffiles%2Fakamai%2Fmobile%2Fimages%2Fandroidsandbox_720.png
Also everything which is beyond the 16th decimal is pretty useless
Floating point details are tricky but…
A 32-bit floating point to bits calculator shows that “1023000f / 5000000f” is represented as: 0.2046000063419342
0.125 * 1.6368000507354736 = 0.2046000063419342
So the extra bits past …342 imply to me that you are converting, casting or doing something even more subtle since dividing the doubles “1023000.0 / 5000000.0” would’ve had a different result.
Best thing to do is read up on floating point. Bruce Dawson’s blog is pretty educational: Floating Point | Random ASCII – tech blog of Bruce Dawson
