Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid

Re: shocking imprecision

by davido (Archbishop)
on Oct 17, 2011 at 22:23 UTC ( #932033=note: print w/replies, xml ) Need Help??

in reply to shocking imprecision

It's only shocking because you grew up with base-ten and are so accustomed to seeing the problem that you don't notice it until it turns up in another base system.

...and because you haven't read perlfaq4: Why am I getting long decimals....., and perlfaq4: Why is int() broken?

Some explanation: Rational numbers do not always have a terminating expansion for a given base. In base ten, we are so accustomed to seeing 1/3rd expressed as .33 (maybe with more significant digits) that we hardly even think about it. But the fact is that .33333333333 is not equal to 1/3rd. You cannot represent 1/3rd in a finite number of digits using a decimal (base ten) representation. You could say that 1/3rd has an infinite expansion, or a non-terminating expansion in base ten. In base ten, any rational number that can be represented as k/((2^n)*(5^m)) would be a number that has a terminating expansion. 4.39 is 439/((2^2)*(5^2)), which fits that formula nicely, and so we get a terminating expansion from 439/100 to 4.39.

In base two (binary), the same applies, but with a different formula; some numbers cannot be represented in a finite number of binary digits. It turns out that in base two, any rational number with a denominator that is a power of two will have a terminal expansion. But all other rational numbers will not. So n/2, n/4, n/8, n/16, n/32, n/64... those could all be represented in binary format in a finite number of binary digits. Or to put it in terms normalized to the previous formula, k/2^n would represent a number that has a terminating expansion in base 2. But 4.39 is 439/100's, which is a rational number that can not be represented in binary with a finite number of digits.

That being the case, computer does what we do when we see 1/3rd in base ten; it approximates. It stores the number as some binary value that is as close as possible to 4.39 in binary format. Just as .33 is slightly less than 1/3rd, 4.39 stores as slightly less than 4.39.

So the computer stores 4.39 as 4.38999999999999. What happens when you ask for 4.38999999999999 * 100? You get 438.99999999999. And when you ask for the integer value of 438.9999999999, you get 438.

This is not a Perl problem. You would be equally shocked by the "imprecision" of Basic, Pascal, C, Modula-II, C++, and just about every other language that doesn't attempt to hide the truth from you.

An illustration: Try this little exercise:

perl -MO=Deparse -e 'print 4.39 * 100' perl -MO=Deparse -e 'print int( 4.39 * 100 )'

The output should be something like:

print 438.99999999999994;


print 438;

So in the first example, the multiplication is indeed storing the number as something slightly less than 439, but because print quietly rounds that to 439 you don't notice. However, int wouldn't be doing a very good job if it quietly rounded when you asked for truncation (which is what int is supposed to do). So it's taking that 438.9999999999999994 and truncating the floating point portion, just like it's supposed to.

Other reading: