Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change

Re^3: Reading wrong value from excel sheet

by space_monk (Chaplain)
on May 20, 2013 at 20:10 UTC ( #1034398=note: print w/replies, xml ) Need Help??

in reply to Re^2: Reading wrong value from excel sheet
in thread Reading wrong value from excel sheet

Your calculations will still round up if you accept 5 or 6 decimal places, From your examples It is only the least significant bit at 14-15dp or so that is causing the problem

1.139999999999999 will round up to 1.14 at anything up to 14dp (if I've counted correctly)

If you spot any bugs in my solutions, it's because I've deliberately left them in as an exercise for the reader! :-)
  • Comment on Re^3: Reading wrong value from excel sheet

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1034398]
[oiskuu]: Prepare your piano with tuna and your cat will show you some love.
thezip has a sudden compusion to dance the tango...
[robby_dobby]: Likewise! I haven't seen a lot of people with HAs back where I come from, so I figured it might be the same around here too...
[robby_dobby]: Oh, shit -- I was replying to someone, but it wound up here. -_-

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (14)
As of 2017-11-17 19:33 GMT
Find Nodes?
    Voting Booth?
    In order to be able to say "I know Perl", you must have:

    Results (272 votes). Check out past polls.