in reply to Integers sometimes turn into Reals after substraction
First of all, 4/25 is periodic in binary (just like 1/3 is periodic in decimal).
____________________ 4/25 = 0.00101000111101011100 base 2
This means it can't be stored exactly in a floating point number.
Sometimes, floating point number errors get canceled out or reduced to nothing. It turns out that this happens for one of the chains of operations but not the other.
That's why you want to avoid checking if two floating point numbers are equal without allowing for an error margin. Replace
$n1 == $n2
abs($n1 - $n2) < $epsilon
|Replies are listed 'Best First'.|
Re^2: Integers sometimes turn into Reals after substraction (error correction ?)
by LanX (Sage) on May 14, 2016 at 15:54 UTC
What's puzzling me is this behaviour:
interestingly this only seems to happen near some powers of 2
in hindsight this may be an effect of precision and correction of the underlying processor and usage of arithmetic units, hence machine dependent.
by pryrt (Monsignor) on May 14, 2016 at 17:02 UTC
IEEE double-precision float is 53 bit float. 0.00000000002910383046 = 2**-35. With the 18 bits for the integral portion of 255160 or 256160, that's the full 53bits. So, apparently, when you take the 53bit fractional representation of 255.16 multiplied by 1000dec, round-off sets the final bit to 0; when you take 256.16 * 1000, round-off sets the final bit to 1. Now (when I have time) I'm going to have to figure out the formula to figure out which powers of two will have this error and which won't. It all depends on the 53rd bit of the product of 2**N+16/25 times 128 (same as the bits of times 1000, since the factor of 8 is a binary shift). Oh, I'm nearly there... the 125 is 0b111_1101... and that will just interact with the 53rd bit... urgh but I've got to leave now... I guess it's an exercise for later. :-)
by pryrt (Monsignor) on May 14, 2016 at 19:21 UTC
Convert ieee floats into hex notation, for each of 2**n - 1 and 2**n: