Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

How fast is pp/PAR::Packer at packing DateTime::Format::ISO8601 for you?

by Anonymous Monk
on Oct 18, 2012 at 12:02 UTC ( #999712=perlquestion: print w/ replies, xml ) Need Help??
Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

If you have pp/PAR::Packer, how long does this take?

perl -le " print ~~gmtime " pp -e " use strict; use warnings; use Getopt::Long; use DateTime::Form +at::ISO8601; print qq{$_\n} for %INC; " perl -le " print ~~gmtime "

On my old laptop it took 32min at 98% cpu usage

It doesn't seem to get faster (on second run) even after adding  -cd pp-scandeps-cachefile   which has about 2084 files

I thought, hey, the cache does md5summing, all that md5summing is taking extra time, so I produced this patch

diff -ruN Module-ScanDeps-1.09/lib/Module/ScanDeps/Cache.pm Module-Sca +nDeps-1.09-new/lib/Module/ScanDeps/Cache.pm --- Module-ScanDeps-1.09/lib/Module/ScanDeps/Cache.pm 2012-02-21 07 +:25:42.000000000 -0800 +++ Module-ScanDeps-1.09-new/lib/Module/ScanDeps/Cache.pm 2012-10-1 +8 04:31:16.312500000 -0700 @@ -57,7 +57,7 @@ } ### check for existence of the entry -### check for identity of the file +### check for identity of the file, ONLY if mtime is different ### pass cached value in $mod_aref ### return true in case of a hit @@ -66,6 +66,14 @@ my ($key, $file, $mod_aref) = @args{qw/key file modules/}; return 0 unless (exists $cache->{$key}); my $entry = $cache->{$key}; + + my $mtime = ( stat $file )[9]; + if (exists $entry->{mtime} and $entry->{mtime} == $mtime ){ + ## if mtime hasn't changed, don't bother with md5, its faster + @$mod_aref = @{$entry->{modules}}; + return 1; + } + my $checksum = _file_2_md5($file); if ($entry->{checksum} eq $checksum){ @$mod_aref = @{$entry->{modules}}; @@ -81,6 +89,7 @@ my $checksum = _file_2_md5($file); $entry->{checksum} = $checksum; $entry->{modules} = [@$mod_aref]; + $entry->{mtime} = ( stat $file )[9]; ## cause md5 is slow $cache_dirty = 1; return 1; } diff -ruN Module-ScanDeps-1.09/lib/Module/ScanDeps.pm Module-ScanDeps- +1.09-new/lib/Module/ScanDeps.pm --- Module-ScanDeps-1.09/lib/Module/ScanDeps.pm 2012-09-04 13:38:32 +.000000000 -0700 +++ Module-ScanDeps-1.09-new/lib/Module/ScanDeps.pm 2012-10-18 04:3 +3:48.578125000 -0700 @@ -4,7 +4,7 @@ use warnings; use vars qw( $VERSION @EXPORT @EXPORT_OK @ISA $CurrentPackage @Includ +eLibs $ScanFileRE ); -$VERSION = '1.09'; +$VERSION = '1.0901'; @EXPORT = qw( scan_deps scan_deps_runtime ); @EXPORT_OK = qw( scan_line scan_chunk add_deps scan_deps_runtime path +_to_inc_name );

But that doesn't seem to speed anything up (I killed it after 9min)

Ideas?

Comment on How fast is pp/PAR::Packer at packing DateTime::Format::ISO8601 for you?
Select or Download Code
Re: How fast is pp/PAR::Packer at packing DateTime::Format::ISO8601 for you?
by marto (Bishop) on Oct 18, 2012 at 12:10 UTC
    C:\>perl -le " print ~~gmtime " Thu Oct 18 12:06:43 2012 C:\>pp -e " use strict; use warnings; use Getopt::Long; use DateTime:: +Format::IS O8601; print qq{$_\n} for %INC; " C:\>perl -le " print ~~gmtime " Thu Oct 18 12:07:42 2012

    My laptop setup is Intel Core2Duo @2.1GHz, 1GB RAM, Windows XP SP3, Strawberry Perl 5.12.3.

    Update: I didn't notice that Radia was running, which usually slows down the laptop a fair bit. I'll retest once it's done.

    Updated: results from running the above after Radia stops.

    C:\>perl -le " print ~~gmtime " Thu Oct 18 12:43:16 2012 C:\>pp -e " use strict; use warnings; use Getopt::Long; use DateTime:: +Format::IS O8601; print qq{$_\n} for %INC; " C:\>perl -le " print ~~gmtime " Thu Oct 18 12:43:39 2012
Re: How fast is pp/PAR::Packer at packing DateTime::Format::ISO8601 for you?
by daxim (Chaplain) on Oct 18, 2012 at 12:43 UTC
    $ time pp -e " use strict; use warnings; use Getopt::Long; use DateTime::Format::ISO8601; print qq{$_\n} for %INC; "
    
    real    0m11.527s
    user    0m6.153s
    sys     0m0.900s
    
    I suspect something else is slowing down file access on your system. Use Sysinternals Filemon or similar to detect this.

      I suspect something else is slowing down file access on your system. Use Sysinternals Filemon or similar to detect this.

      I have, it doesn't appear to :) Though I will schedule some downtime later to run chkdsk etc

        The program runs in ~1 second (perl -e ... ) like you'd expect, its the packing that takes forever -- if something else was slowing things down, the running would take forever also

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://999712]
Approved by marto
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (8)
As of 2015-07-04 19:17 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (60 votes), past polls