Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Callback multiplexer

by Dallaylaen (Scribe)
on Apr 04, 2013 at 15:31 UTC ( #1027002=perlquestion: print w/ replies, xml ) Need Help??
Dallaylaen has asked for the wisdom of the Perl Monks concerning the following question:

Suppose I have a number of tasks in one application which may finish in any order. And I need to run some code when all of the tasks have finished. If that matters, the application is running under AnyEvent, but without Coro.

To some extent, AnyEvent's $cv->begin/$cv->end allow what I want. However, I'd like to have more fine-grained control. For instance, I'd like to be unable to "finish" a task twice. Ability to gather data from all tasks would also be nice.

Of course, this can be done. Set up lots of callbacks that share a hash; delete keys from that hash whenever a task finishes; call the megacallback when the hash is empty. I wonder if there's a more civilized way of doing it, maybe some CPAN module?

Comment on Callback multiplexer
Re: Callback multiplexer
by Dallaylaen (Scribe) on Apr 05, 2013 at 07:32 UTC
    I'd like to be able to write something like this:
    #!/usr/bin/perl -w use strict; use 5.010; use Some::Module; # Set goals my $cb = Some::Module->new( sub { say 'BOOM!' } ); $cb->begin( qw(foo bar) ); # Much later, as tasks start getting done $cb->end( foo => 42 ); # "return" value from task 'foo' $cb->begin( 'baz' ); # can add more tasks, why not $cb->end( 'bar' ); # just finish task 'bar' # still waiting for 'baz' to finish at this point # These shall not pass $cb->end( foo => 41 ); # not a second time $cb->end( food => 'bard' ); # we didn't expect that # Not fond of dying in async code, so probably # add some customizable error handler, # say $cb->on_error(sub {...}); or like that # Finally, last hanging task is done $cb->end( baz => 137 ); # BOOM! # at this point, sub {}->( { foo=>42, bar=>undef, baz=>137 } ) has bee +n called
    Looks easy enough to implement, but maybe there's already something like that?
Re: Callback multiplexer
by Loops (Hermit) on Apr 05, 2013 at 13:00 UTC
    Hi there,

    Not 100% sure on everything you're looking for or how you're expecting things to work. But maybe this little example will at least get you going:

    #!perl -w use strict; use warnings; use feature 'say'; use AnyEvent::Util; my $spawn_complete = AnyEvent->condvar; sub spawn { my $name = shift; my $sleep = rand 10; $spawn_complete->begin; fork_call { say "$name($$) about to sleep for $sleep"; sleep $sleep; } sub { say "$name Awake!"; $spawn_complete->end; }; } # Run 5 long processes in the background spawn $_ for qw(abc def ghi jkl mno); # Wait for all background tasks to finish $spawn_complete->recv; say "All done";

      Yes, using begin/end is a possible solution. For instance, Twiggy is using logic similar to this.

      However, I would like to be protected from a callback firing twice. I would also like to untie the application logic from condvar handling - it's much easier to develop modules when they just rely on a running event loop.

        There are many possible answers and solutions to address your concerns. And I am new to AnyEvent myself. So could you explain in the example above how the callback could fire twice?

        Notice too that in the example, all of the condvar handling is confined to the parent process. The callback only gets called when the child finishes and returns. All of the application logic exist in child processes and has no knowledge of condvars. This alows you to run code from say CPAN, that was written with no intention of being executed this way. And to my untrained eye, no danger of a callback double fire.

Re: Callback multiplexer
by salva (Monsignor) on Apr 05, 2013 at 17:46 UTC
    Of course, this can be done. Set up lots of callbacks that share a hash; delete keys from that hash whenever a task finishes; call the megacallback when the hash is empty. I wonder if there's a more civilized way of doing it, maybe some CPAN module?

    I find this approach pretty civilized. I am sure any module providing that functionality will actually do it in this same way under the hood.

Re: Callback multiplexer
by Dallaylaen (Scribe) on Apr 06, 2013 at 20:00 UTC

    Crossposted to stackoverflow and got a reply that satisfied me.

    The author, LeoNerd, suggests using either Future or Async::MergePoint which both fullfill my needs - Future in a more generalized way, and MergePoint exactly in the form I imagined when writing my post.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1027002]
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (4)
As of 2014-07-13 09:10 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    When choosing user names for websites, I prefer to use:








    Results (248 votes), past polls