Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

Re: Reading wrong value from excel sheet

by space_monk (Chaplain)
on May 20, 2013 at 17:34 UTC ( #1034377=note: print w/ replies, xml ) Need Help??


in reply to Reading wrong value from excel sheet

Looks like a rounding error - can't you format the results to show values in the needed precision?

If you spot any bugs in my solutions, it's because I've deliberately left them in as an exercise for the reader! :-)


Comment on Re: Reading wrong value from excel sheet
Re^2: Reading wrong value from excel sheet
by sandeep_car (Novice) on May 20, 2013 at 17:55 UTC

    Hi space_monk,

    Thnks for your repply. This happens only with some record like in 1,000 records only with 5-6 record. So can't format it or do round off again as this will affect other value.I dont want other value to be rounded off. There are values which have values upto 3 places or 4 places.

    thanks,

    Sandeep

      Your calculations will still round up if you accept 5 or 6 decimal places, From your examples It is only the least significant bit at 14-15dp or so that is causing the problem

      1.139999999999999 will round up to 1.14 at anything up to 14dp (if I've counted correctly)

      If you spot any bugs in my solutions, it's because I've deliberately left them in as an exercise for the reader! :-)

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1034377]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (11)
As of 2014-08-20 18:42 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (121 votes), past polls