Behaviour of int() unexpected

ceade1000 has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: Behaviour of int() unexpected by pryrt (Abbot) on Mar 10, 2025 at 21:33 UTC
Internally, all floating points are represented as a binary fraction (effectively), and there is no binary fraction exactly equal to 8.95. For a double-precision floating-point (which is what NV often are in modern perl binaries; see `perl -V:nvsize` ), 8.95 is internally closer to 8.9499999999999992894572642399, per this calculator. 100 times that will be slightly less than 895.0, which means that int and floor both truncate it down, whereas printing only 6 digits after the decimal will round up to 895. What Every Computer Scientist Should Know About Floating-Point Arithmetic Also, if your perl is new enough, the `%a` or `%A` sprintf conversion will show the hex version of the internal floating-point binary representation used: `linux shell: perl -f -e 'printf qq(%A\n), $_ for 8.95, 8.95100, 895' windows cmd.exe: perl -f -e "printf qq(%A\n), $_ for 8.95, 8.95100, 895"` [download] Mine shows: `0X1.1E66666666666P+3 0X1.BF7FFFFFFFFFFP+9 0X1.BF8P+9` [download] update: the anonymous post slightly beat me to the punch, though showing a single-precision float value, rather than the double-precision float that is more likely for your perl to be using. In this instance, they both are slightly less than 8.95 originally, so both round to 894 after your multiplication; but in some cases, the single and double-precision values fall on different sides of the rouding boundary, so rounding a float and rounding a double give different values. Everyone who is ever going to program a floating-point application should watch Superman III. (Okay, that's showing my age; slightly more recent is Office Space.)	[reply] [d/l] [select]
Re^2: Behaviour of int() unexpected by ikegami (Patriarch) on Mar 11, 2025 at 00:31 UTC
The way I like to explain it is that 895/100 is periodic in binary (and hex), just like 1/3 is periodic in decimal. (That's why you see the repeating 6s.) As such, storing it accurately as a floating point number would take infinite amount of storage. The closest floating point number is used instead. In this case, that's 0x1.1E66666666666 * 2^3 = 8.949999999999999289457264239899814128875732421875.	[reply]
Re^2: Behaviour of int() unexpected by harangzsolt33 (Deacon) on Mar 11, 2025 at 00:33 UTC
Internally, all floating points are represented as a binary fraction (effectively) Interesting. To me, this makes no sense at all. Up to this point, I belived that a float point value such as 8.95 is stored as a plain integer like 895 along with an exponent which is -2. So, to turn that into the original number, you print "895" and then move the decimal point to the left by two spaces. But that's apparently not how it's done. But this would be the most logical way to encode a float number if you ask me. Convert 8.95 to 8.9499999999999992894572642399 and store it like that. Who came up with this "standard" and WHY?????	[reply]
Re^3: Behaviour of int() unexpected by ikegami (Patriarch) on Mar 11, 2025 at 00:41 UTC
Oh, and how do you imagine 1/3 being stored in your model? There's no pair of integers m and e where m * 10^e gives 1/3. Your model doesn't work for periodic numbers. You could approximate it. You could use m = 3333333333 and e = -10, but it's not quite exact. And guess what. That's exactly the problem faced here. The number is stored as two integers. But it's not a power of 10; it's a power of 2. So, in this case, you need a pair of integers m and e where m * 2^e = 895/100. There is no such pair of integers because 895/100 is periodic in binary. So the computer used m = 0x11E66666666666 and e = -49, but it's not quite exact. `$ perl -Mv5.14 -e'say 8.95 == 0x11E66666666666 * 2*(-49) ? 1 : 0' 1 $ perl -Mv5.14 -e'say sprintf "%.751g", 0x11E66666666666 2*(-49)' 8.949999999999999289457264239899814128875732421875 $ perl -Mv5.14 -e'say sprintf "%.751g", 8.95' 8.949999999999999289457264239899814128875732421875` [download] If you want to be picky, an IEEE double is actually stored as follows to get a couple of extra bits: ( -1 )^s ( 1 + ( m * 2^( -52 ) ) * 2^( e - 1023 ) )	[reply] [d/l]
Re^4: Behaviour of int() unexpected by harangzsolt33 (Deacon) on Mar 11, 2025 at 02:41 UTC
Re^5: Behaviour of int() unexpected by LanX (Saint) on Mar 11, 2025 at 03:59 UTC
Some notes below your chosen depth have not been shown here
Re^5: Behaviour of int() unexpected by ikegami (Patriarch) on Mar 11, 2025 at 03:25 UTC
Re^3: Behaviour of int() unexpected by syphilis (Archbishop) on Mar 11, 2025 at 03:30 UTC
But this would be the most logical way to encode a float number if you ask me. What you describe is essentially what Math::BigFloat does. If you use Math::BigFloat, then you won't be disappointed by the results you get. (Performance is comparatively sluggish - though perhaps not to an extent that it becomes an issue.) Cheers, Rob	[reply]
Re^3: Behaviour of int() unexpected by LanX (Saint) on Mar 11, 2025 at 03:48 UTC
> plain integer like 895 along with an exponent which is -2. That's decimal logic not binary. Number crunching in decimal is a waste of digital resources.� And 10 is not a universal constant, just the number of the long things we use to hold our hamburgers. See also: Humans have too many fingers for a more detailed discussion. �) Unless you invent affordable micro transistors with 10-ary states. Cheers Rolf _{(addicted to the Perl Programming Language :) see Wikisyntax for the Monastery}	[reply]
Re^4: Behaviour of int() unexpected by cavac (Prior) on Mar 13, 2025 at 15:27 UTC
Re^5: Behaviour of int() unexpected by LanX (Saint) on Mar 13, 2025 at 20:25 UTC
Some notes below your chosen depth have not been shown here
Re: Behaviour of int() unexpected by Anonymous Monk on Mar 10, 2025 at 21:32 UTC
What Every Programmer Should Know About Floating-Point Arithmetic: https://floating-point-gui.de/ 8.95 is stored as 8.94999980926513671875 and int rounds down.	[reply]
Re^2: Behaviour of int() unexpected by ikegami (Patriarch) on Mar 11, 2025 at 00:19 UTC
Perl normally uses `double` for floating point numbers, even on 32-bit machines. You can find out what your `perl` uses as follows: `$ perl -V:nvtype nvtype='double';` [download] A `double` is an IEEE double-precision number on a desktop computer. As such, 8.95 is actually stored as 0x1.1E66666666666 * 2^3 = 8.949999999999999289457264239899814128875732421875.	[reply] [d/l] [select]
Re: Behaviour of int() unexpected by hippo (Archbishop) on Mar 10, 2025 at 21:55 UTC
It's an FAQ. 🦛	[reply]
Re: Behaviour of int() unexpected by syphilis (Archbishop) on Mar 10, 2025 at 22:41 UTC
But why does it round 895.000000 to 894? As others have explained, it's not actually doing that. I just wanted to point out that what you get depends upon the precision of perl's NV (floating point type). If perl's NV is the 64-bit precision long double, then you'll get what you expected: `D:\>perl -le "$x = 8.95 * 100; print int($x);" 895` [download] Otherwise you get what you did not expect: `D:\>perl -le "$x = 8.95 * 100; print int($x);" 894` [download] (For some precisions, the closest binary approximation of 8.95 is less than 8.95, for other precisions it's greater than 8.95.) To determine perl's NV type, just run: `D:\>perl -V:nvtype nvtype='long double';` [download] Cheers, Rob	[reply] [d/l] [select]
Re^2: Behaviour of int() unexpected by pryrt (Abbot) on Mar 10, 2025 at 23:47 UTC
syphilis said, 64-bit precision long double I checked again, just to make sure: `C:\usr\local\share\github\notepad-plus-plus>perl -V:nvtype nvtype='double'; C:\usr\local\share\github\notepad-plus-plus>perl -V:nvsize nvsize='8';` [download] Double is the 64 bit (8 byte) precision. "Long double" doesn't seem to have much consistentency in the WP article (80, 96, or 128 bits) (update: though re-reading, I now see that GNU C seems to have chosen 80bit, so that's what I'm now guessing your nvsize will show). I don't have access to a perl with nvtype='long double', so could you double-check the nvsize on your "long double" version?	[reply] [d/l]
Re^3: Behaviour of int() unexpected by syphilis (Archbishop) on Mar 11, 2025 at 03:02 UTC
could you double-check the nvsize on your "long double" version? Yes, it's the 80-bit extended precision long double - 1 bit for the sign, 15 bits for the exponent, and 64 bits for the mantissa. If it were the IEEE-754 long double, then the one-liner would have returned 894, as for "double" and "__float128". If you want to find out which of the various "long double" formats is being honored by your perl, then (assuming perl is not more than about 10 years old) you can run `perl -V:longdblkind`, which will return a value in the range `-1..9`. The meaning of the returned value is explained in the Config docs: "longdblkind" From d_longdbl.U: This variable, if defined, encodes the type of a long double: 0 = double, 1 = "IEEE" 754 128-bit little endian, 2 = "IEEE" 754 128-bit big endian, 3 = x86 80-bit little endian, 4 = x86 80-bit big endian, 5 = double-double 128-bit little endian, 6 = double-double 128-bit big endian, 7 = 128-bit mixed-endian double-double (64-bit LEs in "BE"), 8 = 128-bit mixed-endian double-double (64-bit BEs in "LE"), 9 = 128-bit "PDP"-style mixed-endian long doubles, -1 = unknown format +. [download] Cheers, Rob	[reply] [d/l] [select]
Re: Behaviour of int() unexpected by sectokia (Friar) on Mar 11, 2025 at 02:43 UTC
Some people have told you why, but not what to do about it. In general case of a 32bit float the float has a factor of 23bit accuracy, and since base 10 log of 2^23 = 6.9 digits, it is safe to use 5 digits for yourself, and 1 digit for error. For example. If you know your numbers are 000.00 to 999.99 then you can simply add 0.001 before you round down (or subtract 0.001 before you round up), then you will never encounter this problem and your numbers will always have their 'ideal' value. `perl -e "my $val = 8.95; printf('%d',int(($val + 0.001) * 100));" 895` [download] If you have 64bit float then its safe to assign 14 digits for yourself and use the 15th for error:	[reply] [d/l]
Re^2: Behaviour of int() unexpected by ForgotPasswordAgain (Vicar) on Mar 11, 2025 at 17:45 UTC
Or use '%.0f' instead of '%d': `perl -Mstrict -wE'my $val = 8.95; printf("%.0f\n", 100*$val);'`	[reply] [d/l]
Re^3: Behaviour of int() unexpected by ikegami (Patriarch) on Mar 12, 2025 at 16:18 UTC
That rounds rather than truncate.	[reply]
Re: Behaviour of int() unexpected by karlgoethebier (Abbot) on Mar 17, 2025 at 11:02 UTC
2�: `#!/usr/bin/env perl use strict; use warnings; use feature qw(say); use Math::Round; say 8.95 * 100; printf("%.15f\n", 8.95 * 100); printf("%f\n", 8.95 * 100); printf("%d\n", 8.95 * 100); printf("%d\n", round(8.95 * 100)); say int(8.95 * 100); say int(round(8.95 * 100)); __END__` [download] `karl@rantanplan:~/src/cpp/acme$ ./meditation.pl 895 894.999999999999886 895.000000 894 895 894 895` [download] `#include <iomanip> #include <iostream> #include <cmath> using std::cout; using std::round; int main() { double a = 8.95; double b = 100; double r = a * b; cout.precision(18); cout << r << " - precision 18\n"; cout << int(r) << " - int \n"; cout << round(r) << " - round\n"; cout.precision(6); cout << r << " - precision 6\n"; cout << "Supernatural!\n"; return 1; }` [download] `karl@rantanplan:~/src/cpp/acme$ clang++-18 -std=c++23 meditation.cpp - +o meditation karl@rantanplan:~/src/cpp/acme$ ./meditation 894.999999999999886 - precision 18 894 - int 895 - round 895 - precision 6 Supernatural!` [download] perlnumber �The Crux of the Biscuit is the Apostrophe�	[reply] [d/l] [select]


Just another Perl shrine
	PerlMonks