Re: Benchmarking Quiz

[Arguile]

"Premature optimisation is the root of all evil." -- Donald Knuth

Knuth's famous quotation applies to many of these constructs. Especially as readability is often never included in such benchmarks :). I was glad to see your warning at the top of the quiz but I thought I'd add my own.

Question 9-10 in particular can be very misleading; especially given that you don't show your method. As it's showing a counting loop, instead of an array iteration loop, it could easily trap the unwary. See Efficient Looping Constructs for some good discussion on this.

I'm also a little inclined to question your methodology. Are you using subs with lexicals to make sure they don't have any interaction with each other? I'm not saying you're doing anything wrong, just some of these results suprised me and don't compare with my benchmarks. This could just be differences in our approach and choice of test strings.

As a side note, consider for ($i=1; $i < $n+1; $i++). You're doing an extra assignment in there every loop. That slows it down considerably for ($i=1; $i <= $n; $i++) is quite a bit faster (and clearer IMO, though not as clear as Perl-ish for). Seeing a foreach along the lines of for my $i (1..$n) might have been nice too, and the results might suprise you :).

Counting loops have already been covered, but I can't seem to find a good node comparing C style for(;;) loops against Perl style foreach when iterating over lists/arrays (probably their most common usage). So read on if you want a quick benchmark.

I'm not sure how much, if any, good the inital mem grab does. I thought I detected a difference in the speed the first test ran at higher numbers (and therefore less iterations) but I could be wrong. If a more senior monk could advise on whether my thinking is faulty in this, it would be appreciated.

#!/usr/bin/perl -w
use strict;
use Benchmark qw(cmpthese);

# Yes I do want it to be 5000 if $ARGV[0] is 0 ;)
my $n = $ARGV[0] || 5000;

# Grab a chunk of memory and give it to perl.
my @array = (1..50000);
undef @array;

cmpthese(-10, {
    a => "&a($n)",
    b => "&b($n)",
    c => "&c($n)",
    d => "&d($n)",
});

sub a {
    my @a = (1..shift);
    my $m;
    
    for ( @a ) {
        $m = $_;
    };
}

sub b {
    my @a = (1..shift);
    my $m;

    for my $i ( @a ){
        $m = $i
    };
}


sub c {
    my @a = (1..shift);
    my $m;

    $m = $_ for ( @a );
}


sub d {
    my @a = (1..shift);
    my $m;

    for (my $i=0; $i <= $#a; $i++) {
        $m = $a[$i];
    };
}

I leave the running of this as an exercise to the reader. Note that -10 in cmpthese() means that each sub will run aproximately 10CPU seconds. To learn how to benchmark your own code, visit turnstep's excellent tutorial

Update: Yes I know I could have just passed a copy of an array, but it's throw away code and copy->paste had already worked it's magic ;).