Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Re: Getting stranger values in subtraction

by MidLifeXis (Monsignor)
on Feb 08, 2016 at 13:37 UTC ( [id://1154625]=note: print w/replies, xml ) Need Help??


in reply to Getting stranger values in subtraction

This is covered often at PM. It has to do with how the language and machine represent floating point numbers (which is why the Perl6 comment elsewhere in this thread is quite interesting :-) ). Floating point numbers are represented as summations of various powers of two -- a binary state machine can only represent things made up of powers of two. The size of your floating point structure and how it is divided between the value and the exponent determine how many digits of precision you have.

For example, .25 can be represented exactly given one digit for the value (and a couple of more for the exponent) because .25 == 25/100 == 1/4 == 2-2. .75 == 75/100 == 3/4 == sum(2-1, 2-2), and so on.

You have been pointed to the CS documentation on the topic above. If at all possible, when summing financial calculations, use scaled values so that you are only dealing with integers. Rounding errors, or specifically, the skimming off of rounding errors, is the subject of a few (urban legend?) scams over the years.

--MidLifeXis

  • Comment on Re: Getting stranger values in subtraction

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1154625]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others lurking in the Monastery: (5)
As of 2024-04-25 14:32 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found