Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask
 
PerlMonks  

Re: Adding parallel processing to a working Perl script

by sundialsvc4 (Abbot)
on Apr 21, 2014 at 13:57 UTC ( #1083021=note: print w/ replies, xml ) Need Help??


in reply to Adding parallel processing to a working Perl script

And, what if you simply opened up several Terminal windows and ran a single command-line command, one per volume (or whatever), in each of those windows?   Instead of taking an already-working script and “parallelizing it,” could you not by other means simply run instances of that unmodified script “in parallel,” using command line parameters to distinguish them?


Comment on Re: Adding parallel processing to a working Perl script
Re^2: Adding parallel processing to a working Perl script
by Jim (Curate) on Apr 21, 2014 at 15:06 UTC

    Thank you for your reply, sundialsvc4.

    Yes, this is the sort of thing I'm doing now.

    start "01" cmd /c perl CountFilesRecords.pl "ABCD/foo bar/"A*/Z* 1^> 0 +1.csv 2^> 01.txt start "02" cmd /c perl CountFilesRecords.pl "ABCD/foo bar/"B*/Z* 1^> 0 +2.csv 2^> 02.txt start "03" cmd /c perl CountFilesRecords.pl "ABCD/foo bar/"C*/Z* 1^> 0 +3.csv 2^> 03.txt start "04" cmd /c perl CountFilesRecords.pl "ABCD/foo bar/"D*/Z* 1^> 0 +4.csv 2^> 04.txt start "05" cmd /c perl CountFilesRecords.pl "ABCD/foo bar/"E*/Z* 1^> 0 +5.csv 2^> 05.txt start "06" cmd /c perl CountFilesRecords.pl "ABCD/foo bar/"F*/Z* 1^> 0 +6.csv 2^> 06.txt start "07" cmd /c perl CountFilesRecords.pl "ABCD/foo bar/"G*/Z* 1^> 0 +7.csv 2^> 07.txt start "08" cmd /c perl CountFilesRecords.pl "ABCD/foo bar/"H*/Z* 1^> 0 +8.csv 2^> 08.txt start "09" cmd /c perl CountFilesRecords.pl "ABCD/foo bar/"I*/Z* 1^> 0 +9.csv 2^> 09.txt start "10" cmd /c perl CountFilesRecords.pl "ABCD/foo bar/"J*/Z* 1^> 1 +0.csv 2^> 10.txt

    Blech!

    It's precisely what I want to automate using Perl. It seems to me the best way to solve the problem is to add parallel processing of the folders and files inside the counting Perl script itself.

        Thanks again, zentara.

        Ugh. The whole reason I chose to use Capture::Tiny (after much research) is because it purports to solve all the problems that all the other methods of forking and exec-ing on Windows do not. I read its author's presentation titled How (Not) to Capture Output in Perl.

        I certainly hope I don't have to abandon using Capture::Tiny now. After all, it's working brilliantly. What's failing is something related to my use of Parallel::ForkManager.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1083021]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (7)
As of 2014-11-29 02:37 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My preferred Perl binaries come from:














    Results (203 votes), past polls