Re: RFC: Large Floating Point Numbers

This method fixes 0.000035 rounding down issue.

What issue do you have with 0.000035 rounding to 0.00003 when you've explicitly asked for 5 dp?

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

Comment on Re: RFC: Large Floating Point Numbers - Rounding Errors

Replies are listed 'Best First'.
Re^2: RFC: Large Floating Point Numbers - Rounding Errors by GAVollink (Novice) on Sep 08, 2011 at 14:42 UTC
It is not consistent: 0.000005 up 0.00001 0.000015 up 0.00002 0.000025 up 0.00003 0.000035 dn 0.00003 0.000045 up 0.00005 0.000055 up 0.00006 0.000065 dn 0.00006 0.000075 dn 0.00007 0.000085 up 0.00009 0.000095 up 0.00010	[reply]
Re^3: RFC: Large Floating Point Numbers - Rounding Errors by BrowserUk (Patriarch) on Sep 08, 2011 at 15:36 UTC
Very strange. I'd almost say that you'd discovered a bug in perl or the underlying C-libraries that was causing the double precision (64-bit) NV value to be transitioned through a single precision (32-bit) float whilst being formatted. If the values were always being manipulated as doubles, this shouldn't happen. It could be a that the IEEE rounding mode isn't being set (correctly?). Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice.	[reply]
Re^4: RFC: Large Floating Point Numbers - Rounding Errors by salva (Canon) on Sep 08, 2011 at 16:10 UTC
This is not a bug but the result of converting numbers between base 10 and 2. The problem is that 0.000005, 0.000015, 0.000025, 0.000035, 0.000045, 0.000055, 0.000065, ... can not be represented precisely in base 2: `printf("%.5f %.30f\n", $_, $_) for (0.000005, 0.000015, 0.000025, 0.00 +0035, 0.000045, 0.000055, 0.000065)` [download]	[reply] [d/l]
Re^5: RFC: Large Floating Point Numbers - Rounding Errors by BrowserUk (Patriarch) on Sep 08, 2011 at 16:13 UTC
Re^5: RFC: Large Floating Point Numbers - Rounding Errors by BrowserUk (Patriarch) on Sep 08, 2011 at 16:18 UTC
Re^6: RFC: Large Floating Point Numbers - Rounding Errors by salva (Canon) on Sep 08, 2011 at 16:33 UTC
Some notes below your chosen depth have not been shown here


Do you know where your variables are?
	PerlMonks

Re: RFC: Large Floating Point Numbers - Rounding Errors