Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Re: Bad math

by Marshall (Prior)
on Oct 31, 2011 at 15:04 UTC ( #934899=note: print w/ replies, xml ) Need Help??


in reply to Bad math

A floating point number will be internally represented by a mantissa and an exponent. This is of course in base 2. The fractional parts of the numbers you show cannot be represented exactly in base 2. Hence there are more binary digits than there are decimal digits and some precision is lost in the conversion process. This is normal. Your solution is the right way to do this.


Comment on Re: Bad math

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://934899]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (5)
As of 2014-08-28 04:54 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (256 votes), past polls