Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?

Re: Bad math

by Marshall (Prior)
on Oct 31, 2011 at 15:04 UTC ( #934899=note: print w/ replies, xml ) Need Help??

in reply to Bad math

A floating point number will be internally represented by a mantissa and an exponent. This is of course in base 2. The fractional parts of the numbers you show cannot be represented exactly in base 2. Hence there are more binary digits than there are decimal digits and some precision is lost in the conversion process. This is normal. Your solution is the right way to do this.

Comment on Re: Bad math

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://934899]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others having an uproarious good time at the Monastery: (4)
As of 2015-10-07 00:52 GMT
Find Nodes?
    Voting Booth?

    Does Humor Belong in Programming?

    Results (167 votes), past polls