in reply to Re: Integers sometimes turn into Reals after substraction
in thread Integers sometimes turn into Reals after substraction

Well Perl's error correction (or that of the underlying C lib) is better than it's reputation
DB<186> $x1= 1000*(4/25) => 160 DB<187> $x2= 4000/25 => 160 DB<188> $x1 == $x2 => 1

What's puzzling me is this behaviour:

DB<205> for (254..263) {$x = 1000 * ( $_ + 4/25 ); printf "%.20f\n" +, $x} 254160.00000000000000000000 255160.00000000000000000000 256160.00000000002910383046 257160.00000000002910383046 258160.00000000002910383046 259160.00000000002910383046 260160.00000000002910383046 261160.00000000002910383046 262160.00000000000000000000 263160.00000000000000000000

Cheers Rolf
(addicted to the Perl Programming Language and ☆☆☆☆ :)
Je suis Charlie!

update

interestingly this only seems to happen near some powers of 2

DB<220> for (0..20) {$e=2**$_; $x = 1000 * ( $e + 4/25 ); printf "$ +_:$e => %.20f\n", $x if ($x-int($x))} 5:32 => 32159.99999999999636202119 8:256 => 256160.00000000002910383046 9:512 => 512159.99999999994179233909 10:1024 => 1024160.00000000011641532183 11:2048 => 2048159.99999999976716935635 15:32768 => 32768160.00000000372529029846 18:262144 => 262144159.99999997019767761230 19:524288 => 524288160.00000005960464477539 20:1048576 => 1048576159.99999988079071044922

update

in hindsight this may be an effect of precision and correction of the underlying processor and usage of arithmetic units, hence machine dependent.

Replies are listed 'Best First'.
Re^3: Integers sometimes turn into Reals after substraction (error correction ?)
by pryrt (Monsignor) on May 14, 2016 at 17:02 UTC

    IEEE double-precision float is 53 bit float. 0.00000000002910383046 = 2**-35. With the 18 bits for the integral portion of 255160 or 256160, that's the full 53bits. So, apparently, when you take the 53bit fractional representation of 255.16 multiplied by 1000dec, round-off sets the final bit to 0; when you take 256.16 * 1000, round-off sets the final bit to 1. Now (when I have time) I'm going to have to figure out the formula to figure out which powers of two will have this error and which won't. It all depends on the 53rd bit of the product of 2**N+16/25 times 128 (same as the bits of times 1000, since the factor of 8 is a binary shift). Oh, I'm nearly there... the 125 is 0b111_1101... and that will just interact with the 53rd bit... urgh but I've got to leave now... I guess it's an exercise for later. :-)

      Convert ieee floats into hex notation, for each of 2**n - 1 and 2**n:

      cmd.exe> perl -e "use 5.010; for(0..31){$e=2**$_; $f=$e-1; $str = join + '', map { sprintf '%02x ', ord } split(//, pack('dddd', $f+4/25, $e+ +4/25, ($f+4/25)*1000, ($e+4/25)*1000)); $str =~ s/(.{24})(.{24})(.{24 +})(.{24})/\1\t\2\t\3\t\4/; say qq{$_\t$str};}" 0 7b 14 ae 47 e1 7a c4 3f 8f c2 f5 28 5c 8f f2 3f + 00 00 00 00 00 00 64 40 00 00 00 00 00 20 92 40 1 8f c2 f5 28 5c 8f f2 3f 48 e1 7a 14 ae 47 01 40 + 00 00 00 00 00 20 92 40 00 00 00 00 00 e0 a0 40 2 48 e1 7a 14 ae 47 09 40 a4 70 3d 0a d7 a3 10 40 + 00 00 00 00 00 b0 a8 40 00 00 00 00 00 40 b0 40 3 a4 70 3d 0a d7 a3 1c 40 52 b8 1e 85 eb 51 20 40 + 00 00 00 00 00 f8 bb 40 00 00 00 00 00 e0 bf 40 4 52 b8 1e 85 eb 51 2e 40 29 5c 8f c2 f5 28 30 40 + 00 00 00 00 00 9c cd 40 00 00 00 00 00 90 cf 40 5 29 5c 8f c2 f5 28 3f 40 14 ae 47 e1 7a 14 40 40 + 00 00 00 00 00 6e de 40 ff ff ff ff ff 67 df 40 6 14 ae 47 e1 7a 94 4f 40 0a d7 a3 70 3d 0a 50 40 + 00 00 00 00 00 d7 ee 40 00 00 00 00 00 54 ef 40 7 0a d7 a3 70 3d ca 5f 40 85 eb 51 b8 1e 05 60 40 + 00 00 00 00 80 0b ff 40 00 00 00 00 00 4a ff 40 8 85 eb 51 b8 1e e5 6f 40 c3 f5 28 5c 8f 02 70 40 + 00 00 00 00 c0 25 0f 41 01 00 00 00 00 45 0f 41 9 c3 f5 28 5c 8f f2 7f 40 e1 7a 14 ae 47 01 80 40 + 00 00 00 00 e0 32 1f 41 ff ff ff ff 7f 42 1f 41 10 e1 7a 14 ae 47 f9 8f 40 71 3d 0a d7 a3 00 90 40 + 00 00 00 00 70 39 2f 41 01 00 00 00 40 41 2f 41 11 71 3d 0a d7 a3 fc 9f 40 b8 1e 85 eb 51 00 a0 40 + 00 00 00 00 b8 3c 3f 41 ff ff ff ff 9f 40 3f 41 12 b8 1e 85 eb 51 fe af 40 5c 8f c2 f5 28 00 b0 40 + 00 00 00 00 5c 3e 4f 41 00 00 00 00 50 40 4f 41 13 5c 8f c2 f5 28 ff bf 40 ae 47 e1 7a 14 00 c0 40 + 00 00 00 00 2e 3f 5f 41 00 00 00 00 28 40 5f 41 14 ae 47 e1 7a 94 ff cf 40 d7 a3 70 3d 0a 00 d0 40 + 00 00 00 00 97 3f 6f 41 00 00 00 00 14 40 6f 41 15 d7 a3 70 3d ca ff df 40 ec 51 b8 1e 05 00 e0 40 + 00 00 00 80 cb 3f 7f 41 01 00 00 00 0a 40 7f 41 16 ec 51 b8 1e e5 ff ef 40 f6 28 5c 8f 02 00 f0 40 + 00 00 00 c0 e5 3f 8f 41 00 00 00 00 05 40 8f 41 17 f6 28 5c 8f f2 ff ff 40 7b 14 ae 47 01 00 00 41 + 00 00 00 e0 f2 3f 9f 41 00 00 00 80 02 40 9f 41 18 7b 14 ae 47 f9 ff 0f 41 3d 0a d7 a3 00 00 10 41 + 00 00 00 70 f9 3f af 41 ff ff ff 3f 01 40 af 41 19 3d 0a d7 a3 fc ff 1f 41 1f 85 eb 51 00 00 20 41 + 00 00 00 b8 fc 3f bf 41 01 00 00 a0 00 40 bf 41 20 1f 85 eb 51 fe ff 2f 41 8f c2 f5 28 00 00 30 41 + 00 00 00 5c fe 3f cf 41 ff ff ff 4f 00 40 cf 41 21 8f c2 f5 28 ff ff 3f 41 48 e1 7a 14 00 00 40 41 + 00 00 00 2e ff 3f df 41 01 00 00 28 00 40 df 41 22 48 e1 7a 94 ff ff 4f 41 a4 70 3d 0a 00 00 50 41 + 00 00 00 97 ff 3f ef 41 00 00 00 14 00 40 ef 41 23 a4 70 3d ca ff ff 5f 41 52 b8 1e 05 00 00 60 41 + 00 00 80 cb ff 3f ff 41 00 00 00 0a 00 40 ff 41 24 52 b8 1e e5 ff ff 6f 41 29 5c 8f 02 00 00 70 41 + 00 00 c0 e5 ff 3f 0f 42 00 00 00 05 00 40 0f 42 25 29 5c 8f f2 ff ff 7f 41 14 ae 47 01 00 00 80 41 + 00 00 e0 f2 ff 3f 1f 42 ff ff 7f 02 00 40 1f 42 26 14 ae 47 f9 ff ff 8f 41 0a d7 a3 00 00 00 90 41 + 00 00 70 f9 ff 3f 2f 42 00 00 40 01 00 40 2f 42 27 0a d7 a3 fc ff ff 9f 41 85 eb 51 00 00 00 a0 41 + 00 00 b8 fc ff 3f 3f 42 00 00 a0 00 00 40 3f 42 28 85 eb 51 fe ff ff af 41 c3 f5 28 00 00 00 b0 41 + 00 00 5c fe ff 3f 4f 42 01 00 50 00 00 40 4f 42 29 c3 f5 28 ff ff ff bf 41 e1 7a 14 00 00 00 c0 41 + 00 00 2e ff ff 3f 5f 42 ff ff 27 00 00 40 5f 42 30 e1 7a 94 ff ff ff cf 41 71 3d 0a 00 00 00 d0 41 + 00 00 97 ff ff 3f 6f 42 01 00 14 00 00 40 6f 42 31 71 3d ca ff ff ff df 41 b8 1e 05 00 00 00 e0 41 + 00 80 cb ff ff 3f 7f 42 ff ff 09 00 00 40 7f 42 n 2**n - 1 2**n + (2**n - 1)*1000 (2**n) * 1000