Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask
 
PerlMonks  

Re: How to make perl script work simltaneously on multiple objects

by McA (Priest)
on Mar 29, 2013 at 14:52 UTC ( #1026157=note: print w/replies, xml ) Need Help??


in reply to How to make perl script work simltaneously on multiple objects

Hi,

I would go with the fork option. You seem to have a process running which detects new server instances. This would be your main loop (pseudocode):

my %found; my %active; # sighandler for sigchild $SIG{'SIGCHLD'} = sub { my $pid = waitpid; delete $active{$pid}; }; while(running) { if(my $serverid = new_server_found()) { $found{$serverid} = { 'name' => $servername, 'otherinfo' => $otherinfo, }; my $pid = fork; if(defined $pid) { # fork ok if($pid == 0) { # this is in the child do_all_stuff_necessary_for_hardening(); exit $rc; } else { # this is parent # You have the pid and you know that this # fork was concerning a server you found # probably you want to store this information $active{$pid} = $serverid; } else { # no fork => error handling die ("FATAL: Something went wrong"); } } sleep(1); }
BE AWARE: This is perlish pseudocode and should give only hints.

McA

  • Comment on Re: How to make perl script work simltaneously on multiple objects
  • Download Code

Replies are listed 'Best First'.
Re^2: How to make perl script work simltaneously on multiple objects
by slayedbylucifer (Scribe) on Apr 01, 2013 at 04:10 UTC
    Thanks McA. Sorry for late reply. I will evaluate fork usage and will see how it goes. I not sure how much code change would it take.

      Hi,

      if you fork a perl script which does have all to process the job it should be quite easy as you don't need to exec and you do have everything in hands. Anything you set up before fork is also accessible in the child. Be careful. Memory usage is multiplied.

      There a some points you have to be aware of:

      • Childs inherit open io handles. If you don't need them close them in childs.
      • If you have a DBI handle the childs inherit it. The DESTROY handler of DBI is shutting down the connection when childs exit. So look at manpage of DBI concerning DBI and forking.
      • Make a SIGCHLD handler for reaping of just for setting flags to reap in parent. waitpid is your friend. Sometimes it's nice to call the non-blocking version of waitpid.
      • Be aware that the SIGCHLD handler interrupts a sleep(), so that this sleep may be much less than the amount of seconds specified.

      McA

        Hi McA,

        I had a developer colleague of mine help me on writing below fork code (after following your advice).

        #!/usr/bin/perl -w use strict; use POSIX; my @list=("server1","server2","server3","server4","server5","server6", +"server7","server8"); my $count = 0; my $pcount = 5; foreach (@list) { chomp ; my $real_host = $_; if( (my $pid = fork()) == 0) { print "Processing - $_ \n"; my $wait=ceil(rand(10)); print "Wait:$wait \n"; sleep($wait); print "Exit Status= $? \n"; if($? ==0) { print "$_ Exit successfully \n"; } else { print "$_ Does not Exit successfully \n"; } print "Completing - $_ \n"; exit; } $count++; while($count >= $pcount) { wait(); $count--; } } while(wait() != -1){}

        I still donít understand few parts of this code but I am OK with that. This test code is working perfectly. So I am planning to use the same logic in my orchestrator.

        Many thanks for your help.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1026157]
help
Chatterbox?
[Corion]: marioroy: Oh, that's always cool, having API-compatible modules. This makes testing and comparing things much easier
[marioroy]: IPC in MCE::Shared can handle 400k (sends) per second. That's seems a lot for being a pure-Perl module. After making the release, will come back and post a solution for a node by a fellow wanting faster logging.
[Corion]: While working on WWW::Mechanize:: Chrome, I had the suspicion that AnyEvent was doing something wrong, but I was able to swap it out for Mojolicious and the error persisted.
[Corion]: Of course, the error was in my own code ;)
[marioroy]: Corion, start and start_child in MCE::Hobo::Manager return a MCE::Hobo object, whereas P::FM returns the PID. I can have it return the PID though. I tried Hobo::Manager with several P::FM modules, just changed P::FM to MCE::Hobo::Manager and it works.
[marioroy]: I also have a Hobo driver for Forklift allowing folks to use in multiple classes, no conflicts with one another. That's not possible for P::FM.

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (5)
As of 2017-05-26 08:38 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?