Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Multiple system commands in parallel

by g_speran (Beadle)
on May 09, 2016 at 19:52 UTC ( #1162570=perlquestion: print w/replies, xml ) Need Help??
g_speran has asked for the wisdom of the Perl Monks concerning the following question:

Hello Perl Monks,
I am looking for a way to launch multiple system commands in parallel. I do not want to wait for the command to finish, I am not interested in the child pid, nor the child output.

When I launch the code below, the scripts waits for the command to finish and then proceeds.

I have looked in perls system and exec. But neither of these provide the functionality I need.

Any thoughts?

$NSRHOLD="c:/temp/hold/"; @files=List_Hold(); $MAXCNT=15; for ($x=0; $x < scalar(@files); $x++ ) { $file=@files[$x]; chomp $file; $cmd="clone -F $file "; print "Launching Command is: $cmd\n"; system($cmd); } sub List_Hold { my $dir = "$NSRHOLD" . "*"; my @files = <"$dir">; return @files; }

Replies are listed 'Best First'.
Re: Multiple system commands in parallel
by BrowserUk (Pope) on May 09, 2016 at 21:24 UTC

    If you use:

    for ($x=0; $x < scalar(@files); $x++ ) { $file=@files[$x]; chomp $file; $cmd="start /min clone -F $file "; print "Launching Command is: $cmd\n"; system($cmd); }

    The programs will be started asynchronously, running in separate process group (with their own minimised windows), and will neither wait for the processes to complete nor be susceptible to ^C in the parent session.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re: Multiple system commands in parallel
by Tanktalus (Canon) on May 09, 2016 at 20:07 UTC

    Since you're on windows, if you use system(1,$cmd) I think that'll spawn it off.

    On unix, you'd have to fork and exec instead.

    You just have to wait for them to finish if you're going to read files they create.

    Personally, I use AnyEvent::Util::run_cmd to spawn subprocesses, monitor their outputs and return code, and manage all of that. But that's a lot more than you're asking for.


      Tanktalus....Thank you...this has gotten me one step closer.

      Unfortunately, I noticed a small issue. In the event there are 15 files in the directory that need to be process and the script has currently launched, say 8. If the user <ctrl><c> out of the script prior to the script exiting on its own, the 8 that the script initially launch are killed when the user <ctrl><c> out of the script.

      Any ideas on how to rectify this?

        I'm not 100% sure about what's required on Windows. What may work is to set $SIG{INT} to "IGNORE" before spawning the children. Note that when you do so, your process won't be killed by ctrl-C, either - you'll just keep going.

        Another option may be to fork/exec yourself instead of using system, and in the "child" process, do the above - that way the parent process will still be killable partway through while the children will continue to live on their own, e.g.:

        for ($x=0; $x < scalar(@files); $x++ ) { $file=@files[$x]; chomp $file; $cmd="clone -F $file "; print "Launching Command is: $cmd\n"; my $pid = fork; if ($pid) { # parent } elsif (defined $pid) { $SIG{INT} = 'IGNORE'; exec($cmd); } else { die "fork failed."; } }
        This variation you can still hit Ctrl-C to kill the parent, but the children will inherit the ignore status on the signal handler and keep going. (This can also be done in the on_prepare callback in AnyEvent::Util::run_cmd.)

        Finally, if you have control over the clone command, you can have that program ignore the INT signal.

        I ran "help start" on Win XP. I presume that there are more options on other Window's O/S's. The "/B" option appears to do what you want? (ignore CTL-C?)
Re: Multiple system commands in parallel
by Your Mother (Bishop) on May 09, 2016 at 22:08 UTC

    I personally would reach for Parallel::ForkManager and Capture::Tiny; and most certainly add use strict; and use warnings; :P Plenty of approaches and if this is a throwaway script, without a need to speed-limit, a simple fork might do.

      Under Windows, both P::FM and fork use threads; which doesn't address the OPs problem.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
      In the absence of evidence, opinion is indistinguishable from prejudice.

        The problem seems to be synchronous v asynchronous. Do threads run synchronously in WIN when paired with system? Is all that is necessary to use exec which Tanktalus already mentioned and I had, perhaps wrongly assumed, was already part of the advice here?

Re: Multiple system commands in parallel
by Marshall (Abbot) on May 11, 2016 at 02:28 UTC
    I hope that this will present another option... launch Tk without a command Window. My thinking is that if there is no command window at all, there there is no way to send ^C to that program from the console.
Re: Multiple system commands in parallel
by Anonymous Monk on May 10, 2016 at 13:07 UTC

    Thank you all for the comments. I have tried, system, fork, and start but they all are not producing the necessary result on windows....

    All of the attempts kill the child process when <ctrl><c> is issued prior to the loop finishing. In addition, when using fork, if the loop finished successfully, the parent scripts sits and waits for the last child to complete and then the parent script exits.

      Which version of Windows are you using and what code did you write?

      I ask this because on Windows 10, the following works perfectly:

      a.pl:

      use 5.16.2; system "start perl a.plx"; system "start perl a.plx"; system "start perl a.plx"; my $i = 1; say $i++ while sleep 1;

      a.plx:

      use 5.16.2; my $i = 1; say $i++ while sleep 1;

      The reason this works in Windows is because start spawns a new command prompt instance, completely detached from the parent instance. This separate instance can be terminated separately with ctrl-c, so if you're looking for something that is unkillable, you will have to look elsewhere (such as firing the code under a different user account, where the logged in user has no rights to terminate software running under the alternate account)


        Windows 7

        Im just trying to have it basically spawn a separate process in the background. defiantly dont need a separate command window that it opens in. I even tried 'start /B'. but even that gets killed if the parent is <ctrl><c> out of.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1162570]
Approved by Tanktalus
help
Chatterbox?
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (6)
As of 2018-07-16 22:48 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    It has been suggested to rename Perl 6 in order to boost its marketing potential. Which name would you prefer?















    Results (352 votes). Check out past polls.

    Notices?