noticeable numerical error

Hi Tuan,

Matt’s analysis is correct. The only thing I’ll add is a bit about binary representation of floating point numbers. When I taught Intro to Computer Science back in Grad school, I’d usually take a full lecture explaining it. I’ll limit myself here.

“0.1” can’t be represented exactly in floating point representation. Hence, you’re using an approximate value. The more bits used in the floating point representation (i.e the precision Single, Double, Quad) the closer the approximation but it’s still ‘wrong’. The more operation performed on the value, the larger the amount of error.

The all knowing Wikipedia has a good write-up on Binary that may be useful: Binary number - Wikipedia

  • Mat