Re: Benchmarks for module compilation time

[simonm]

Very nice, thanks!

How would you feel about posting your benchmarking code so we can try this on other platforms?

[itub]

There was not really code as such, just a quick and dirty series of shell commands. It went something like this:

#!/bin/bash
 
modules="Archive::Tar Archive::Zip CGI Class::DBI DBI Data::Dumper Lin
+gua::EN::Inflect Math::Trig POSIX Template XML::SAX XML::Simple YAML 
+diagnostics strict warnings"

export TIMEFORMAT="%R"
rm -f table
for m in $modules; do
    time perl -M$m -e1;
    for i in `seq 1 10`; do
        time perl -M$m -e1;
    done &> $m
    echo -ne "$m\t" >> table
    awk '{t+=$1} END{print t/10}' $m >> table
done
 
sort -k2 -rn table > table.sorted

The part that computes "all" and "none" is left as an exercise to the reader. ;-)

I hope I won't be downvoted for using shell instead of Perl! ;-) Sadly, this makes the test less portable.

[radiantmatrix]

Hm, it wouldn't be very hard to do this with Perl. I can think of two approaches: either invoke perl via system(), or use string-based eval.

The following can be used like:
# this_script.pl [--eval] [--perl /my/perl/interpreter] module module2 module3

use strict;
use warnings;
use Time::HiRes;
use Benchmark ':hireswallclock',':all';
use Getopt::Long;

my $PERL = 'C:\\Perl\bin\perl.exe';
my $use_eval = 0;

GetOptions ( 'perl=s' => \$PERL, 'eval' => \$use_eval );

my %testhash = map { 
   $_ => ( $use_eval ? "eval 'use $_'" : "system('$PERL', '-M$_ ', '-e
+1')" )
} @ARGV;

timethese( ($use_eval ? 10000 : 1000), \%testhash );

This one was created to work with ActiveState Perl on Windows. Pass the --perl parameter to use it on other platforms.

Update:itub confirms my suspicions that eval isn't as accurate a test as the system route. Thanks for the explanation of why, itub! It's worth noting that the system call is the default approach in the code above.

Updates:

• 2005-07.Jul-26 : clarified preferred use of system over eval, thanks to itub

[itub]

Thanks for the port. :) It's not that I wrote a shell script; it was all done interactively. What I posted here was a reconstruction from looking at the shell history.

I don't think the 'eval' version is working properly, bacause the second time you call eval for the same module, it becomes effectively a no-op. You can make perl forget that it already included a module by deleting its entry from %INC (for example, delete $INC{'CGI.pm'}). That doesn't clean up the package, however, which sometimes leads to spurious warnings, and some modules may do strange things when their initialization code is run twice. It is trickier to delete the dependencies as well, to return to a "blank slate", but it should be possible. Maybe there's even a CPAN module for that. ;-)