Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Re: Parallel processing with ForkManager

by zentara (Archbishop)
on Sep 20, 2012 at 11:27 UTC ( #994636=note: print w/ replies, xml ) Need Help??


in reply to Parallel processing with ForkManager

I haven't been able to find good examples of using ForkManager to read a file a line-at-a-time and then doing something with those lines.

This isn't exactly your scenario, but it may help get you going.

#!/usr/bin/perl use warnings; use strict; use Parallel::ForkManager; my $dir = shift || '.'; my @dirs = get_sub_dirs($dir); my $max_tasks = 3; my $pm = new Parallel::ForkManager($max_tasks); $|++; my $start = time(); for my $dir (@dirs) { my $pid = $pm->start and next; printf "Begin processing $dir at %d secs.....\n", time() - $start; #push all the $dir/files into @ARGV and search through them #line by line @ARGV = <$dir/*>; while (<ARGV>) { close ARGV if eof; #find some search term, using "perl" for example if( $_ =~ /perl/) { print "$ARGV: $. :$_\n"; $pm->finish; goto END; } } END: printf ".... $dir done at %d secs!\n", time() - $start; $pm->finish; } print " all done\n"; exit; ########################################################## sub get_sub_dirs { my $dir = shift; opendir my $dh, $dir or die "Error: $!"; my @dirs = grep { -d $_ } readdir $dh; @dirs = grep !/^\.\.?$/, @dirs; closedir $dh; return @dirs; }

I'm not really a human, but I play one on earth.
Old Perl Programmer Haiku ................... flash japh


Comment on Re: Parallel processing with ForkManager
Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://994636]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (8)
As of 2014-09-19 21:44 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (147 votes), past polls