Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

Receiving data from many children

by xiper (Friar)
on Aug 20, 2003 at 02:47 UTC ( #285088=perlquestion: print w/ replies, xml ) Need Help??
xiper has asked for the wisdom of the Perl Monks concerning the following question:

I have a parent that spawns a bunch of children, up to 10 at any one time, to do some 'mostly-idle' processing. My question is, how to i get the results back from each child? Previously i've used the exit status of the child, which is fine for simple return values, but now i have a complex-data-structure to be returned (although can easily be encoded into a string by Data::Dumper if necessary). I've experimented with pipes but they only seem to work with 1 parent - 1 child relationship. IPC::Shareable looks to do what i want, but i wanted to get input from others to see if there was a simpler solution.

My code so far, if anyone's interested:

my( %pid ); foreach my $num ( "01" .. "25" ) { # if max processes, wait for one to finish if( keys( %pid ) >= 10 ) { &reap() } if( my $pid = fork() ) # parent { $pid{$pid} = $num; } else # child { &child( $num ); exit 0; } } # reap remaining processes while( %pid ) { &reap() } # <collect data> sub reap { my( $pid ) = wait(); if( $pid == -1 ) { die( "no more kids to reap!\n" ) } if(! $pid{$pid} ) { die( "why did I see $pid?\n" ) } delete( $pid{$pid} ); } sub child { my( $num ) = $_[0]; print "child $num spawned\n"; # <send data> }

Thanks all!

- ><iper

my JAPH: print"Just another Perl hacker\n"; # ^ look, no space! pretty tricky huh?

Comment on Receiving data from many children
Select or Download Code
Re: Receiving data from many children
by Zaxo (Archbishop) on Aug 20, 2003 at 03:09 UTC

    You can open a pipe before you start spawning the children. Each child closes the read side of the pipe, and prints data to the write side. The parent reads from the pipe, which has become shared among the kids by duplication of the handle.

    I published a similar example here at Many-to-One pipe. There, however, I had one child collecting the data from the others. You can adapt that to your requirements.

    After Compline,
    Zaxo

      Hmm, i tried experimenting with pipes before i posted but couldn't get them to work... Just tried again, copying your example, and *tada*, works! Finally figured out that in my first attempt i had the reader/writer mixed up in the initial pipe... *slaps self*

      Future readers, the end result:

•Re: Receiving data from many children
by merlyn (Sage) on Aug 20, 2003 at 03:16 UTC
Re: Receiving data from many children
by Flame (Deacon) on Aug 20, 2003 at 16:06 UTC

    You could also try the Threads implementation if you're running 5.8.0. There, you can simply return values and the parent can get them via the join command. True, Threads are still kinda experemental...

    Hey, it's your call...



    My code doesn't have bugs, it just develops random features.

    Flame ~ Lead Programmer: GMS (DOWN) | GMS (DOWN)

Re: Receiving data from many children
by NetWallah (Abbot) on Aug 21, 2003 at 00:02 UTC
    Here is a working sample of multiple threads. Tested on Activeperl 5.8 on Win32.
    use strict; use threads; my @MyThread; my $threadcount = 15; for (0..($threadcount-1)){ $MyThread[$_]=threads->new(\&s,$_,10+$_); }; if ($ARGV[0] eq 'a'){ foreach (@MyThread){ print qq(Return check via thread array from ) . $_->join() . qq(\n +); } }else{ foreach (threads->list()){ print qq(Return check via threads list from ) . $_->join() . qq(\n +); } } ####################### sub s{ my $a=shift; sleep($threadcount-$a); print qq(In sub s param=$a\n); return shift() }

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://285088]
Approved by blue_cowdawg
Front-paged by broquaint
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (11)
As of 2014-12-26 19:34 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (174 votes), past polls