Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Re^7: "Automated" benchmarking

by BrowserUk (Pope)
on Jan 24, 2012 at 14:16 UTC ( #949691=note: print w/ replies, xml ) Need Help??


in reply to Re^6: "Automated" benchmarking
in thread "Automated" benchmarking

Thank you for the concise and clear explanation. But ... :)

Isn't the 'nulltime' compensation code meant to correct for (most of) that?

# "Null time" compensation code $nulltime = 0; for (1..100) { my($u,$s,$cu,$cs) = times; $cstart = $u+$s+$cu+$cs; $start = Time::HiRes::time; &$testDB; ($u,$s,$cu,$cs) = times; $cdone = $u+$s+$cu+$cs; $done = Time::HiRes::time; $diff = $done - $start; $nulltime += $diff; } $nulltime /= 100; ... my($delta); $delta = $done - $start; $delta = ($delta > $nulltime) ? $delta - $nulltime : 0; ...

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

The start of some sanity?


Comment on Re^7: "Automated" benchmarking
Download Code
Re^8: "Automated" benchmarking
by salva (Monsignor) on Jan 24, 2012 at 17:06 UTC

    A nulltime compensation is effective when the overhead is relatively small compared to the sentence execution time. But when it is much bigger as in this case, the compensated overhead (aka error) can still be of the same order of magnitude of the time we want to measure.

    Guess why the last line you have quoted from the source is there!

    package DB; use Time::HiRes; my %testDB = ( null => sub { my($pkg,$filename,$line) = caller; $profile || return; %packages && !$packages{$pkg} && return; }, work => sub { $do_something{foo} = 'bar'; my($pkg,$filename,$line) = caller; $profile || return; %packages && !$packages{$pkg} && return; }); my $n = 10; for (1..$n) { for my $t (keys %testDB) { my $testDB = $testDB{$t}; my $nulltime = 0; for (1..100) { my($u,$s,$cu,$cs) = times; $cstart = $u+$s+$cu+$cs; $start = Time::HiRes::time; &$testDB; ($u,$s,$cu,$cs) = times; $cdone = $u+$s+$cu+$cs; $done = Time::HiRes::time; $diff = $done - $start; $nulltime += $diff; } $time{$t} = $nulltime / 100 * 1e6; } printf "null: %.2f, work: %.2f, delta: %.2f\n", $time{null}, $time{work}, $time{work} - $time{null}; }

      I get this output from your code. Is that good, bad or ugly?

      null: 6.63, work: 7.41, delta: 0.77 null: 6.45, work: 7.60, delta: 1.14 null: 6.41, work: 7.36, delta: 0.95 null: 6.49, work: 7.25, delta: 0.77 null: 6.47, work: 7.41, delta: 0.93 null: 6.52, work: 7.37, delta: 0.85 null: 6.38, work: 7.30, delta: 0.93 null: 6.44, work: 7.22, delta: 0.78 null: 6.49, work: 7.38, delta: 0.89 null: 6.50, work: 7.29, delta: 0.79

      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

      The start of some sanity?

        Well, it seems that on your system the output is pretty consistent as all the delta values are very similar. On my computer it generates a broader range of values:
        null: 4.34, work: 4.53, delta: 0.19 null: 4.70, work: 4.67, delta: -0.03 null: 5.33, work: 6.38, delta: 1.05 null: 4.83, work: 6.05, delta: 1.22 null: 4.68, work: 5.27, delta: 0.58 null: 6.56, work: 5.12, delta: -1.44 null: 4.42, work: 9.26, delta: 4.85 null: 5.26, work: 4.58, delta: -0.68 null: 6.63, work: 5.86, delta: -0.78 null: 4.34, work: 7.54, delta: 3.20

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://949691]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (11)
As of 2014-11-20 21:09 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My preferred Perl binaries come from:














    Results (101 votes), past polls