map, grep, for, foreach

[mt2k]

Okay, I know that this subject has been talked about many times, but I am here to discuss it again!

As my example, I will use reading in data from GET and POST requests in a CGI script.
I will post four example that all produce identical output.
Then I will ask a couple of questions.

What am I asking here? Which is best for simple looping: for, foreach, grep, or map?

The Code Base

#!c:/perl/bin/perl -w

use strict;
use CGI;

my %input;
my $q = new CGI;

# This is where we will insert the four test
# instances. Right under this comment


print "Content-type: text/html\n\n";
(my($n,$v) = each %input) { print "$n: $v<br>"; }

# code changed at mt2k's request.  -kudra Original: 
# print "$n: $v<br>"; while (my($n,$v) = each %input);

Now, look for the comment in that snippet. This is where each one of the following lines could be inserted to read in the query string and STDIN:

Example 1: map { $input{$_} = $q->param($_) } $q->param();

Example 2: grep { $input{$_} = $q->param($_) } $q->param();

Example 3: $input{$_} = $q->param($_) for $q->param();

Example 4: $input{$_} = $q->param($_) foreach $q->param();

Now, the big question. Which one of those four cases above is the most efficient? I am not looking to have "cool looking code". I want to know which one will give the best performance.

Myself, I would say that grep is unnecessary because we are not testing for anything and are not (explicitly) assiging the results to an array. Map also places its results into an array, but in this case should provide better performance than grep would.

I think the real debate I am having here is between for and foreach. It seems to me that for would do a better job, but am I assuming correctly? Also, perhaps map is a better option than either for or foreach...

Wisdom please and thank you!

[rob_au]

How about even more simply and efficiently using methods provided by CGI.pm ...

my %input = CGI::Vars;

The above performs the same as your proposed code but ties your array in with that used internally within CGI.pm for the storage of CGI parameters.

Of the proposed code samples provided however, map and grep would be poor choices as, in the examples given, these are being used in a void context, the return values built and returned by these functions discarded. With regard to for and foreach, according to perlsyn, foreach is merely a synonym for for, the choice between the two representing a balance for the programmer between readability and brevity.

 

[Kanji]

How about even more simply and efficiently using methods provided by CGI.pm ...

        my %input = CGI::Vars;

While it works, Vars was really intended for compatability with the old cgi-lib.pl library, and so such shares the same quirks, like null (\0) seperated values for any multi-valued parameters.

Instead, I'd use CGI's Dump method...

Read more... (697 Bytes)

    --k.

[Anonymous Monk]

As far as I can tell for and foreach are not the same. The do the have the same functionality but due to the internals of perl they execute differently (something to do with lookup tables I believe, but not sure :P ).

A foreach runs far faster than for.

I converted one of my data mangling scripts to foreach loops and the whole thing sped up 35%. Ever since I have stopped believing what they say about for and foreach being the same and only ever use foreach if there is a choice between the two.

[Elian]

I'm interested in how you managed to draw that particular conclusion, since:

  for (1..3) {}

and

  foreach (1..3) {}

compile to the identical same optree.

[jsprat]

C:\>perl -MO=Deparse -e "print for (1..10);print foreach (1..10);"
foreach $_ (1 .. 10) {
    print $_;
}
foreach $_ (1 .. 10) {
    print $_;
}
-e syntax OK

OK, I'm confused. They parse identically.

Do you mean 'C-style' for loops (for (i=1;i++;i<11))vs 'perl-style' for loops? If so, what you describe is documented in perlsyn.

If not, what do you mean?

[runrig]

A foreach runs far faster than for.

Please post a small sample of code demonstrating this phenomenon.

[Abigail-II]

It doesn't matter. Your loop administration isn't the bottleneck, the method call dominates the running time.

But how many parameters do you have anyway? If you look at the total program, is the loop going to be the bottleneck? Optimizing something that only takes a small part of the running time isn't very useful.

Here are some benchmarking results, which show that it's the method call being the bottleneck.

Abigail

       
#!/usr/bin/perl
            
use strict;
use warnings 'all';

use Benchmark::Sized;  # To be released...
use CGI;

use vars qw /$cgi/;

$cgi = CGI -> new;
   
my $stats = sized_timethese
    run_for     =>  1,
    start_size  =>  2,
    end_size    =>  4096,
    steps       =>  12,
    seed        =>  sub {
        $main::cgi -> delete_all;
        foreach (1 .. $_ [0]) {$main::cgi -> param ($_, $_)}
    },
    code        =>  {
        map1    =>
           'my %input = ();
            map  {$input {$_} = $main::cgi -> param ($_)} $main::cgi -
+> param',
        map2    =>
           'my %input =
            map {$_ => $main::cgi -> param ($_)} $main::cgi -> param',
        grep    =>
           'my %input = ();
            grep {$input {$_} = $main::cgi -> param ($_)} $main::cgi -
+> param',
        for     =>
           'my %input = ();
            $input {$_} = $main::cgi -> param ($_) for $main::cgi -> p
+aram'
    }
;

print_timed $stats, type => 'sized';

__END__
Runs/second:
Size       for    grep    map1    map2
   2:  9569.16 9774.55 9773.64 9569.16
   4:  5749.09 5749.09 5749.09 5688.57
   8:  3200.00 3169.81 3169.81 3140.19
  16:  1689.62 1659.26 1644.04 1644.04
  32:   872.73  837.38  845.28  853.33
  64:   439.45  426.67  425.71  430.77
 128:   222.22  216.22  214.29  216.50
 256:   110.19  106.67  106.73  105.71
 512:    53.40   51.38   51.38   50.93
1024:    25.69   24.53   24.53   24.07
2048:    12.50   11.93   11.82   11.54
4096:     6.31    5.94    5.83    5.61

[Zaxo]

Your map and grep are in void context. map does work within the codeblock which could as well be done outside. grep is a plain poor choice. for and foreach are identical:

%input = map { $_ => $q->param($_) } $q->param();

Another possibility is hash slices.

After Compline,
Zaxo

[mt2k]

Okay I screwed that example up... Last line should say:

while (my($n,$v) = each %input) { print "$n: $v<br>"; }

It won't work the way I have it for 2 reasons: 1. I kinda threw a semi-colon in between the EXPR and the loop 2. You must show the loop first when initializing variables within the loop statement (in order to satisfy strict).

And okay, so the following code would work:

#!c:/perl/bin/perl -w
use strict;
use CGI 'Vars';

my %input = CGI::Vars;
print "Content-type: text/html\n\n";
while (my($n,$v) = each %input) { print "$n: $v<br>"; }

Let's go to lala land and pretend CGI.pm could not do this. What would be the best alternative? (By the way, thanks. I have already modified my scripts to use this wonderful CGI::Vars ;))

[Aristotle]

my @param = $q->param();
@input{@param} = map $q->param($_), @param;

I couldn't be bothered to Benchmark it but I suspect this is going to be the most efficient or at least a very competitive variant, especially so if the number of parameters is substantial.

Update: Or maybe this:

my @param = $q->param();
@input{@param}=();
$input{$_} = $q->param($_) for @param;

Finally, this leads us to another variation which I like best:

@input{$q->param()}=();
$input{$_} = $q->param($_) for keys %input;

Between the latter two, I'm not sure which one will perform better.

Makeshifts last the longest.