Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw

Methods for Asynchronous IPC

by athomason (Curate)
on Feb 02, 2001 at 10:53 UTC ( #55958=perlquestion: print w/replies, xml ) Need Help??

athomason has asked for the wisdom of the Perl Monks concerning the following question:

I haven't had need for Unix IPC in a while, so I think there's some easy way to what I need that I'm overlooking. So anything that works is great, but especially clever solutions as always, get bonus points.

I have a load-testing script that makes a specific web request repeatedly until a stop condition is satisfied (the server is supposed to eventually get fed up and stop sending the normal response when too many requests come in from the same IP... and yes, I know that's a terrible way to fend off a DoS). I'd like to be able to run many copies of the script in parallel without actually opening ten xterms and running the script in each. This is easy with fork as long as the children and parent don't need to communicate, but I'd like to have each child report back to the parent periodically saying how many requests it's made so far. The parent would then sum up the requests made from the children and report to stdout on how many requests had been made in total. Execution order of the children isn't relevant.

Looking at perlipc and answers to older questions, I'm thinking about something in the neighborhood of this:

# ... setup code my $NUM_CHILDREN = 10; my $total_count; my @pids; for (my $i = 0; $i < $NUM_CHILDREN; $i++) { $pids[$i] = open(CHILD, "-|"); if ($pids[$i]) { next; } elsif (defined $pid) { # child handler, $pid=0 my $count = 0; my $done = 0; while (!$done) { # make web request; set $done somewhere $count++; } } else { die "Fork failed at number $i: $!\n"; } }
As written, the parent process spawns all the children and exits, which 1) doesn't print hit counts like I want, and 2) doesn't allow me to cancel all the children at once, which I'd like to do. I'm unsure of where and how to send $count up to the parent from the children, and how the parent would make use of that. I'm also not quite sure how to have the parent kill the children when it dies abnormally. Any clues?

Replies are listed 'Best First'.
Re: Methods for Asynchronous IPC
by AgentM (Curate) on Feb 02, 2001 at 11:01 UTC
    You'll need to look into pipes or -if you're pumped- IPC::Shareable. It's nice that you're storing the pids but you don't seem to do anything with them yet. If you're afraid of zombies (who isn't eh?) then you should be waitpidding the kiddies while managing them with your kill directive (infanticide really). A quick and easy hack to make sure these kids go down with the parent is to issue SIGHUP which the programs aren't setup to catch and will thus bombshell. In fact, this is automatically issued when you logout.

    Wait a sec! Is it just me or is this looking like something that LWP::Parallel is supossed to do?

    AgentM Systems nor Nasca Enterprises nor Bone::Easy nor Macperl is responsible for the comments made by AgentM. Remember, you can build any logical system with NOR.
Re: Methods for Asynchronous IPC
by Gloom (Monk) on Feb 02, 2001 at 16:19 UTC
    It can be clever to use Light Weigth Processes ( LWP ). LWP allow you to execute child processes in the same address space than their parents.
    In order to make this, you have to use the clone() syscall.
    This kind of processes can be called "Threads".
    By the way, I don't know how perl implements neither the use of threads ( if it does ) nor the clone syscall.

    I ask the monks : did perl allow multithreaded programmation ?

    Update :

    There is a module called "Thread" that provide you basis functions to implement a parallel processing and shared variables management ( Semaphores... ).
    Main process share global vars with the childs. Each of it can thus freely store a report, and main thread can process it. You may also use signal IPC to inform main process of its child's state.

    Hope this helps ( and I'm not too boring :)

      Perl's threading model is unstable, differs with every major release, and Perl 6 (which is projected to be stable) will undoubtably have yet another model. In short you don't really want to use it.

      Beyond that, the clone() call is very, very much Linux specific. Nobody else does things that way, and if you wish to write portable code in C you won't either.

      Besides which, threaded code is definitely a mixed blessing. There are definitely times when multi-threading is the right thing to do. But I get very wary when I see people pulling it out as an answer everywhere. See Threads vs Forking (Java vs Perl) for further discussion on why.

      Linking by ID seems to be broken at the moment. The missing thing to try for further discussion is Threads vs Forking (Java vs Perl)...

        Very interesting node.
        I see clearer now : thread is not in the "perl" spirit :)
        So I only discover perl and some old c uses reapers sometime !

        Thank's for this answer.

      As far as I know Thread support is still considered as experimental in Perl

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://55958]
Approved by root
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (6)
As of 2022-05-27 18:53 GMT
Find Nodes?
    Voting Booth?
    Do you prefer to work remotely?

    Results (97 votes). Check out past polls.