Re^2: Efficiency of map vs. more verbose basic/fundamental code

[marquezc329]

Thank you for your quick reply and pointing me in the direction of Benchmark. Observant me didn't even think of time passed since the statement was made, but I figured since I asked the question I may as well go ahead and learn how to use Benchmark and get the answer. For anybody interested in the results:

For the sake of simplicity I used the code from the passage broken into subroutines to be called by Benchmark:

#!/usr/bin/perl 
use strict;
use warnings;

use Benchmark qw(:all);

my %h;

@h{'A'..'Z','a'..'z'} = 1..52;

sub verbose {
   my $hash = shift;
   foreach my $key (sort keys %$hash) {
      print "$key:   $hash->{$key}\n";
   }
}

sub idiom {
   my $hash = shift;
   print map "$_: $hash->{$_}\n", sort keys %$hash;
}

timethese(100000, {
   'Verbose' => 'verbose(\%h)',
   'Idiom' => 'idiom(\%h)',
});

timethese(1000000, {
   'Verbose' => 'verbose(\%h)',
   'Idiom' => 'idiom(\%h)',
});

timethese(10000000, {
   'Verbose' => 'verbose(\%h)',
   'Idiom' => 'idiom(\%h)',
});

RESULTS:

---

Benchmark: timing 100000 iterations of Idiom, Verbose...
Idiom: 0 wallclock secs ( 0.08 usr + 0.00 sys = 0.08 CPU) @ 1250000.00/s (n=100000)
(warning: too few iterations for a reliable count)
Verbose: 0 wallclock secs ( 0.12 usr + 0.00 sys = 0.12 CPU) @ 833333.33/s (n=100000)
(warning: too few iterations for a reliable count)

---

Benchmark: timing 1000000 iterations of Idiom, Verbose...
Idiom: 1 wallclock secs ( 0.76 usr + 0.00 sys = 0.76 CPU) @ 1315789.47/s (n=1000000)
Verbose: 1 wallclock secs ( 1.12 usr + 0.00 sys = 1.12 CPU) @ 892857.14/s (n=1000000)

---

Benchmark: timing 10000000 iterations of Idiom, Verbose...
Idiom: 8 wallclock secs ( 7.52 usr + 0.00 sys = 7.52 CPU) @ 1329787.23/s (n=10000000)
Verbose: 12 wallclock secs (11.76 usr + 0.00 sys = 11.76 CPU) @ 850340.14/s (n=10000000)

---

So at least for this particular piece of code map proves to be faster. Thanks again for helping me answer my own question. As always, I'm still a beginner so if this is misguided/messy code or i tested this wrong please let me know.

[ruzam]

Edit: Just so others don't stop here, this benchmark doesn't actually test the functions it claims to. Read the following messages for a more accurate benchmark of map vs foreach. Hint: foreach wins in the end

Hmmm, well now I'm a little surprised. I was going to argue that if you're going to benchmark then you must compare apples to apples. In your example, 'verbose' uses an extra variable 'my $key' which the mapping version does not. I thought that might be a factor, so I made a new function 'verbose2' which uses $_ like the mapping function, expecting that it might be on par (or at least faster than 'verbose').

But using $_ in place of $key was actually slower ???? So I tried one more time with 'verbose3' to re-write the function exactly the same as the mapping version, only using foreach instead. In my mind, verbose2 and verbose3 are exactly the same code and Perl should have interpreted them to be the same at run time, but again verbose3 was slower yet.

Can a Perl innards expert explain why verbose2 is slower than verbose? Or why verbose3 is slower than verbose2?

#!/usr/bin/perl 
use strict;
use warnings;

use Benchmark qw(:all);

my %h;

@h{'A'..'Z','a'..'z'} = 1..52;

sub verbose {
   my $hash = shift;
   foreach my $key (sort keys %$hash) {
      print "$key: $hash->{$key}\n";
   }
}

sub verbose2 {
   my $hash = shift;
   foreach (sort keys %$hash) {
      print "$_: $hash->{$_}\n";
   }
}

sub verbose3 {
   my $hash = shift;
   print "$_: $hash->{$_}\n" foreach sort keys %$hash;
}

sub idiom {
   my $hash = shift;
   print map "$_: $hash->{$_}\n", sort keys %$hash;
}

timethese(1000000, {
   'Verbose' => 'verbose(\%h)',
   'Verbose2' => 'verbose2(\%h)',
   'Verbose3' => 'verbose3(\%h)',
   'Idiom' => 'idiom(\%h)',
});

timethese(10000000, {
   'Verbose' => 'verbose(\%h)',
   'Verbose2' => 'verbose2(\%h)',
   'Verbose3' => 'verbose3(\%h)',
   'Idiom' => 'idiom(\%h)',
});

Results:
Benchmark: timing 1000000 iterations of Idiom, Verbose, Verbose2, Verb
+ose3...
     Idiom:  1 wallclock secs ( 0.85 usr +  0.00 sys =  0.85 CPU) @ 11
+76470.59/s (n=1000000)
   Verbose:  1 wallclock secs ( 1.26 usr +  0.00 sys =  1.26 CPU) @ 79
+3650.79/s (n=1000000)
  Verbose2:  0 wallclock secs ( 1.34 usr +  0.00 sys =  1.34 CPU) @ 74
+6268.66/s (n=1000000)
  Verbose3:  2 wallclock secs ( 1.39 usr +  0.00 sys =  1.39 CPU) @ 71
+9424.46/s (n=1000000)
Benchmark: timing 10000000 iterations of Idiom, Verbose, Verbose2, Ver
+bose3...
     Idiom:  8 wallclock secs ( 8.57 usr +  0.00 sys =  8.57 CPU) @ 11
+66861.14/s (n=10000000)
   Verbose: 12 wallclock secs (12.92 usr +  0.00 sys = 12.92 CPU) @ 77
+3993.81/s (n=10000000)
  Verbose2: 13 wallclock secs (13.06 usr +  0.00 sys = 13.06 CPU) @ 76
+5696.78/s (n=10000000)
  Verbose3: 15 wallclock secs (13.90 usr +  0.01 sys = 13.91 CPU) @ 71
+8907.26/s (n=10000000)

[remiah]

I tried your benchmark and I also see map is faster. When I changed subs to return concatenated results, for loop becomes faster. I wonder why?

#!/usr/bin/perl
use strict; use warnings;
use Benchmark qw/cmpthese/;

my %h = map {$_ => $_} 1 .. 10000;

sub test1{
    my $buff='';
    foreach my $key (sort keys %h) {
        $buff.="$key: $h{$key}\n";
    }
    return $buff;
}
sub test2{
    return join('', map "$_: $h{$_}\n", sort keys %h);
}

print test1() eq test2() ? "same\n" : "not same\n";

my %tests = (
    '01_for' => \&test1,
    '02_map' => \&test2,
);
cmpthese(
    -10, #for 10 cpu secs
    \%tests
);

__DATA__
         Rate 02_map 01_for
02_map 33.9/s     --   -16%
01_for 40.2/s    19%     --

update:
I remember previous thread. "for loop" consumes lots of memory compared to while loop.

[remiah]

hello, again

I noticed $hash arg not passed to functions. You don't see outputs of print statement, do you? Would you try this?



#!/usr/bin/perl
use strict; use warnings;
use Benchmark qw(:all);

my %h;

@h{'A'..'Z','a'..'z'} = 1..52;

sub verbose {
    #my $hash = shift;
    foreach my $key (sort keys %h) {
        print "$key:   $h{$key}\n";
    }
}
sub idiom {
    #my $hash = shift;
    print map "$_: $h{$_}\n", sort keys %h;
}

my %tests = (
    '01_verbose' => \&verbose,
    '02_idiom' => \&idiom,
);

cmpthese(
    -10, #for 10 cpu secs
    \%tests
);

__DATA__
             Rate   02_idiom 01_verbose
02_idiom   5341/s         --        -4%
01_verbose 5577/s         4%         --

Personally, I like map.
regards.

[ruzam]

Edit: thanks remiah for showing me the error of my ways. While updating the benchmark test functions, I also changed the benchmark itself from 'timethese' to 'cmpthese' which displays the results in order of slowest to fastest. Using map is in fact slower than using foreach.

I realized that output should be spewing all over my console with the test right after I hit send (Doh!)

Here's my revised benchmark, with STDOUT redirected to null. Map is still 'not' the clear leader. verbose, verbose2, verbose4 and verbose4 ('for' instead of 'foreach') are all within statistical noise of each other (and can change significantly with each run), so none of them are any better than the other.

Printing to null might also have an affect on the results, but it's clear that map is 'not' faster. Maybe because for/foreach must also be ready for a 'next' or 'last' or even a 'return' within the loop? I may have to re-think when I use foreach vs map.

#!/usr/bin/perl
use strict; use warnings;
use Benchmark qw(:all);

my %h;

@h{'A'..'Z','a'..'z'} = 1..52;

sub verbose {
   my $hash = shift;
   foreach my $key (sort keys %$hash) {
      print "$key: $hash->{$key}\n";
   }
}

sub verbose2 {
   my $hash = shift;
   foreach (sort keys %$hash) {
      print "$_: $hash->{$_}\n";
   }
}

sub verbose3 {
   my $hash = shift;
   print "$_: $hash->{$_}\n" foreach sort keys %$hash;
}

sub verbose4 {
   my $hash = shift;
   print "$_: $hash->{$_}\n" for sort keys %$hash;
}

sub idiom {
   my $hash = shift;
   print map "$_: $hash->{$_}\n", sort keys %$hash;
}

cmpthese(-10, {
   'Verbose' => sub{
       local *STDOUT;
       open STDOUT, '>/dev/null' or warn "Can't open /dev/null: $!";
       verbose(\%h)
       },
   'Verbose2' => sub{
       local *STDOUT;
       open STDOUT, '>/dev/null' or warn "Can't open /dev/null: $!";
       verbose2(\%h)
       },
   'Verbose3' => sub{
       local *STDOUT;
       open STDOUT, '>/dev/null' or warn "Can't open /dev/null: $!";
       verbose3(\%h)
       },
   'Verbose4' => sub{
       local *STDOUT;
       open STDOUT, '>/dev/null' or warn "Can't open /dev/null: $!";
       verbose4(\%h)
       },
   'Idiom' => sub{
       local *STDOUT;
       open STDOUT, '>/dev/null' or warn "Can't open /dev/null: $!";
       idiom(\%h)
       },
});

Results:
            Rate    Idiom  Verbose Verbose4 Verbose3 Verbose2
Idiom    14273/s       --      -9%     -11%     -12%     -12%
Verbose  15703/s      10%       --      -2%      -3%      -3%
Verbose4 16029/s      12%       2%       --      -1%      -1%
Verbose3 16129/s      13%       3%       1%       --      -0%
Verbose2 16151/s      13%       3%       1%       0%       --

[remiah]

"Rate" column shows how many times it could process in one second. Verbose2 is the fastest.

"This chart is sorted from slowest to fastest ..."
see http://search.cpan.org/~rjbs/perl-5.16.1/lib/Benchmark.pm.

How about think of map in conjunction with lisp? Sometimes I see lisp conscious scripts at this perlmonk. For example moritz's max tree depth in this thread, or like this one, recently posted. And several times I saw the name of book 'High Order Perl', which I have not yet read...

regards.