Re: Using Perl to run a Windows command-line utility many times with ordered, parallel execution

Just add the DATA :)

#! perl -slw
use strict;
use threads stack_size => 4096;
use threads::Q;

sub thread {
    my $Q = shift;

    while( my $item = $Q->dq ) {
        system qq[ doit.exe $item ];
    }
}

our $T //= 20;

my @ordered_arguments = <DATA>;
chomp @ordered_arguments;

my $Q = threads::Q->new( $T * 2 );

my @threads = map async( \&thread, $Q ), 1 .. $T;

$Q->nq( $_ ) for @ordered_arguments;
$Q->nq( (undef) x $T );

$_->join for @threads;

__DATA__
...
[download]

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

Comment on Re: Using Perl to run a Windows command-line utility many times with ordered, parallel execution Download Code

Replies are listed 'Best First'.
Re^2: Using Perl to run a Windows command-line utility many times with ordered, parallel execution by jellisii2 (Hermit) on Jan 27, 2014 at 18:10 UTC
So when does threads::Q get published to CPAN? :) I did find it at Re^5: dynamic number of threads based on CPU utilization, though...	[reply]
Re^3: Using Perl to run a Windows command-line utility many times with ordered, parallel execution by BrowserUk (Patriarch) on Jan 27, 2014 at 20:09 UTC
Thread::Queue would work just as well, though it would consume a little more memory without some additional code to limit the size of the queue. But for the OPs 10k items and 20 threads that would still come in at less than 40mb. With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice.	[reply]
Re^4: Using Perl to run a Windows command-line utility many times with ordered, parallel execution by Jim (Curate) on Apr 17, 2014 at 00:35 UTC
What about Thread::Queue::MaxSize?	[reply]
Re^5: Using Perl to run a Windows command-line utility many times with ordered, parallel execution by BrowserUk (Patriarch) on Apr 17, 2014 at 02:30 UTC


Come for the quick hacks, stay for the epiphanies.
	PerlMonks