comment on

I guess I'd say the next steps depend on whether it bothers you (or your end users) to get 1000e-36 instead of 1e-33. If it does, post-processing is probably your best bet -- but I wouldn't use sprintf. I'd look at the number of digits before the "e" (or the decimal point), and if it's four digits, I'd divide by 1000 (and re-round, if necessary) and adjust the exponent. syphilis's suggestion of generating the 1e-33 using Data::Float's hex_float() would work to "fix" your test -- make the test pass; but if you or your end user would consider it wrong to ever see 1000e-36, then forcing a "correct" 1e-33 won't show the actual error.

I was able to make it show 1000e-36 by running

C:\WINDOWS\system32>perl -MNumber::FormatEng=format_pref -le "print fo
+rmat_pref($_) for 0.9999999994e-33, 0.9999999995e-33, 0.9999999996e-3
+3"
999.999999e-36
999.999999e-36
1000e-36
[download]

. Your code $num = 0 + sprintf '%.6f', ($num / $mult); only allows 6 digits after the decimal, and you don't do any more checks after that, so numbers that round from 999.999999x to 1000.000000 are going to show up wrong. In your library, you need to fix that. In your test suite, you need to make sure you test the edge cases of what your code does with that (for example, the %.6f introduces edge cases) -- you don't have to test every representable floating value in binary32, binary64, and binary128, just the edge cases that your specific code introduces. I'd probably do a simple test after that line that if(abs($num)>=1000) { ... $num /= 1000, $e++ } or similar. <code>

In reply to Re^3: Portability of floor(log(N)) by pryrt
in thread Portability of floor(log(N)) by toolic

Are you posting in the right place? Check out Where do I post X? to know for sure.
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
Want more info? How to link or How to display code and escape characters are good places to start.


go ahead... be a heretic
	PerlMonks