Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris
 
PerlMonks  

How fast is pp/PAR::Packer at packing DateTime::Format::ISO8601 for you?

by Anonymous Monk
on Oct 18, 2012 at 12:02 UTC ( [id://999712]=perlquestion: print w/replies, xml ) Need Help??

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

If you have pp/PAR::Packer, how long does this take?

perl -le " print ~~gmtime " pp -e " use strict; use warnings; use Getopt::Long; use DateTime::Form +at::ISO8601; print qq{$_\n} for %INC; " perl -le " print ~~gmtime "

On my old laptop it took 32min at 98% cpu usage

It doesn't seem to get faster (on second run) even after adding  -cd pp-scandeps-cachefile which has about 2084 files

I thought, hey, the cache does md5summing, all that md5summing is taking extra time, so I produced this patch

diff -ruN Module-ScanDeps-1.09/lib/Module/ScanDeps/Cache.pm Module-Sca +nDeps-1.09-new/lib/Module/ScanDeps/Cache.pm --- Module-ScanDeps-1.09/lib/Module/ScanDeps/Cache.pm 2012-02-21 07 +:25:42.000000000 -0800 +++ Module-ScanDeps-1.09-new/lib/Module/ScanDeps/Cache.pm 2012-10-1 +8 04:31:16.312500000 -0700 @@ -57,7 +57,7 @@ } ### check for existence of the entry -### check for identity of the file +### check for identity of the file, ONLY if mtime is different ### pass cached value in $mod_aref ### return true in case of a hit @@ -66,6 +66,14 @@ my ($key, $file, $mod_aref) = @args{qw/key file modules/}; return 0 unless (exists $cache->{$key}); my $entry = $cache->{$key}; + + my $mtime = ( stat $file )[9]; + if (exists $entry->{mtime} and $entry->{mtime} == $mtime ){ + ## if mtime hasn't changed, don't bother with md5, its faster + @$mod_aref = @{$entry->{modules}}; + return 1; + } + my $checksum = _file_2_md5($file); if ($entry->{checksum} eq $checksum){ @$mod_aref = @{$entry->{modules}}; @@ -81,6 +89,7 @@ my $checksum = _file_2_md5($file); $entry->{checksum} = $checksum; $entry->{modules} = [@$mod_aref]; + $entry->{mtime} = ( stat $file )[9]; ## cause md5 is slow $cache_dirty = 1; return 1; } diff -ruN Module-ScanDeps-1.09/lib/Module/ScanDeps.pm Module-ScanDeps- +1.09-new/lib/Module/ScanDeps.pm --- Module-ScanDeps-1.09/lib/Module/ScanDeps.pm 2012-09-04 13:38:32 +.000000000 -0700 +++ Module-ScanDeps-1.09-new/lib/Module/ScanDeps.pm 2012-10-18 04:3 +3:48.578125000 -0700 @@ -4,7 +4,7 @@ use warnings; use vars qw( $VERSION @EXPORT @EXPORT_OK @ISA $CurrentPackage @Includ +eLibs $ScanFileRE ); -$VERSION = '1.09'; +$VERSION = '1.0901'; @EXPORT = qw( scan_deps scan_deps_runtime ); @EXPORT_OK = qw( scan_line scan_chunk add_deps scan_deps_runtime path +_to_inc_name );

But that doesn't seem to speed anything up (I killed it after 9min)

Ideas?

Replies are listed 'Best First'.
Re: How fast is pp/PAR::Packer at packing DateTime::Format::ISO8601 for you?
by marto (Cardinal) on Oct 18, 2012 at 12:10 UTC
    C:\>perl -le " print ~~gmtime " Thu Oct 18 12:06:43 2012 C:\>pp -e " use strict; use warnings; use Getopt::Long; use DateTime:: +Format::IS O8601; print qq{$_\n} for %INC; " C:\>perl -le " print ~~gmtime " Thu Oct 18 12:07:42 2012

    My laptop setup is Intel Core2Duo @2.1GHz, 1GB RAM, Windows XP SP3, Strawberry Perl 5.12.3.

    Update: I didn't notice that Radia was running, which usually slows down the laptop a fair bit. I'll retest once it's done.

    Updated: results from running the above after Radia stops.

    C:\>perl -le " print ~~gmtime " Thu Oct 18 12:43:16 2012 C:\>pp -e " use strict; use warnings; use Getopt::Long; use DateTime:: +Format::IS O8601; print qq{$_\n} for %INC; " C:\>perl -le " print ~~gmtime " Thu Oct 18 12:43:39 2012
Re: How fast is pp/PAR::Packer at packing DateTime::Format::ISO8601 for you?
by daxim (Curate) on Oct 18, 2012 at 12:43 UTC
    $ time pp -e " use strict; use warnings; use Getopt::Long; use DateTime::Format::ISO8601; print qq{$_\n} for %INC; "
    
    real    0m11.527s
    user    0m6.153s
    sys     0m0.900s
    
    I suspect something else is slowing down file access on your system. Use Sysinternals Filemon or similar to detect this.

      I suspect something else is slowing down file access on your system. Use Sysinternals Filemon or similar to detect this.

      I have, it doesn't appear to :) Though I will schedule some downtime later to run chkdsk etc

        The program runs in ~1 second (perl -e ... ) like you'd expect, its the packing that takes forever -- if something else was slowing things down, the running would take forever also

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://999712]
Approved by marto
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others browsing the Monastery: (5)
As of 2024-04-19 15:14 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found