Re^4: RFC: Large Floating Point Numbers - Rounding Errors (expectations)

[tye]

I'll take your rant and raise you one...

Rant? Well, if you want to read it that way, then I can't stop you.

The best answer noted elsewhere in this thread, is a hardware limitation. The SPARC architecture does not have a way to set the rounding mode for floating point operations. However, x86 Operating Systems (including recent MacOS, but still not Solaris) seems to have a way to reach into the floating point hardware to ask for a NEAR rounding mode.

Um, no. I already replied to that claim showing that a much simpler explanation is that the inconsistent rounding is simply a consequence of converting between base 10 and base 2 and that BrowserUk's results are more due to him computing 0.000035 through repeated additions.

I'll be adding a reply shortly that clearly shows that I was correct on that point. The rounding mode makes absolutely no difference in how sprintf rounds 0.000035 and the rounding mode was already at NEAR anyway!

Flippant disregard for naïveté may be geek-chic

Yeah, again, if you want to read it that way, then I can't stop you. I actually tried to give a useful solution for meeting naive 4th-grade expectations:

printf( "%.5f\n", sprintf("%­.12e",$_) )

(except I left off inserting the '1' by mistake, which would complicate the code a bit, but nowhere near as much as your "nobody else could do it efficiently" code.) I decided not to go into it in much detail such as noting that the value for 12 can be somewhat different on different builds. I'd hoped Config exposed the number of digits Perl considered significant but it doesn't seem to. Replace the 12 with a value perhaps equal or slightly less than Perl's "significant digits" count.