Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

Re^3: Converting a parallel-serial shell script

by BrowserUk (Pope)
on Sep 18, 2008 at 12:50 UTC ( #712271=note: print w/replies, xml ) Need Help??


in reply to Re^2: Converting a parallel-serial shell script
in thread Converting a parallel-serial shell script

Updated: This modified version actually terminates. ++Corion

Needs some error checking, but this should work:

#! perl -slw use strict; use threads; use Thread::Queue; our $THREADS ||= 4; my $Qraw = new Thread::Queue; my $Qtsv = new Thread::Queue; sub convert{ $_[ 0 ] } sub toTSV { while( my $filename = $Qraw->dequeue ) { my $outFile = $filename . '.tsv'; open my $fhIn, '<', $filename or warn "$filename : $!" and nex +t; open my $fhOut, '>', $outFile or warn "$outFile : $!" and nex +t; while( <$fhIn> ) { my $tsv = convert( $_ ); print $fhOut $tsv; } close $fhOut; close $fhIn; $Qtsv->enqueue( $outFile ); } $Qtsv->enqueue( undef ); } my @threads = map threads->create( \&toTSV ), 1 .. $THREADS; ## Filenames from command line, 4 threads to terminate $Qraw->enqueue( @ARGV, (undef) x $THREADS ); for( 1 .. $THREADS ) { while( my $tsvFile = $Qtsv->dequeue ) { system "echo $tsvFile"; unlink $tsvFile; } } $_->join for @threads;

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://712271]
help
Chatterbox?
[Corion]: 1nickt: Finding autobox in production would give me pause, yes
[LanX]: efficient survey
[MidLifeXis]: And under MINGW64_NT-6.1 MYHOST 2.6.0(0.304/5/3) 2016-09-09 09:46 x86_64 Msys there seem to be issues with escapes in external build tool calls.
[Corion]: I mean, it's a technical feat it achieves, but... why? ;)
[MidLifeXis]: And it also has the 0.14 version of the tarball in its manifest.
[LanX]: avoiding unreadable brackets
[MidLifeXis]: Although the previous one could be a b0rken PATH, I would need to dig for that.
[thezip]: I've got to go to meetings now. If anyone has further comments regarding Spreadsheet::XLSX deployment to Strawberry Perl 5.24.1, please /msg me -- thanks!
LanX has to go ... plans to crash with a car into a group of pythonistas while screaming "LARRY IS THE GREATEST"
LanX ... darn ... where is my car?

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (13)
As of 2017-03-23 17:26 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Should Pluto Get Its Planethood Back?



    Results (291 votes). Check out past polls.