Callback multiplexer

[Dallaylaen]

Suppose I have a number of tasks in one application which may finish in any order. And I need to run some code when all of the tasks have finished. If that matters, the application is running under AnyEvent, but without Coro.

To some extent, AnyEvent's $cv->begin/$cv->end allow what I want. However, I'd like to have more fine-grained control. For instance, I'd like to be unable to "finish" a task twice. Ability to gather data from all tasks would also be nice.

Of course, this can be done. Set up lots of callbacks that share a hash; delete keys from that hash whenever a task finishes; call the megacallback when the hash is empty. I wonder if there's a more civilized way of doing it, maybe some CPAN module?

[Loops]

Hi there,

Not 100% sure on everything you're looking for or how you're expecting things to work. But maybe this little example will at least get you going:

#!perl -w
use strict;
use warnings;
use feature 'say';
use AnyEvent::Util;

my $spawn_complete = AnyEvent->condvar;

sub spawn
{
        my $name = shift;
        my $sleep = rand 10;
        $spawn_complete->begin;
        fork_call {
                say "$name($$) about to sleep for $sleep";
                sleep $sleep;
        } sub {
                say "$name Awake!";
                $spawn_complete->end;
        };
}

# Run 5 long processes in the background
spawn $_ for qw(abc def ghi jkl mno);

# Wait for all background tasks to finish
$spawn_complete->recv;
say "All done";

[Dallaylaen]

Yes, using begin/end is a possible solution. For instance, Twiggy is using logic similar to this.

However, I would like to be protected from a callback firing twice. I would also like to untie the application logic from condvar handling - it's much easier to develop modules when they just rely on a running event loop.

[Loops]

There are many possible answers and solutions to address your concerns. And I am new to AnyEvent myself. So could you explain in the example above how the callback could fire twice?

Notice too that in the example, all of the condvar handling is confined to the parent process. The callback only gets called when the child finishes and returns. All of the application logic exist in child processes and has no knowledge of condvars. This alows you to run code from say CPAN, that was written with no intention of being executed this way. And to my untrained eye, no danger of a callback double fire.

[Dallaylaen]

I'd like to be able to write something like this:

#!/usr/bin/perl -w

use strict;
use 5.010;

use Some::Module;

# Set goals
my $cb = Some::Module->new( sub { say 'BOOM!' } );
$cb->begin( qw(foo bar) );

# Much later, as tasks start getting done
$cb->end( foo => 42 );       # "return" value from task 'foo'
$cb->begin( 'baz' );         # can add more tasks, why not
$cb->end( 'bar' );           # just finish task 'bar'
# still waiting for 'baz' to finish at this point

# These shall not pass 
$cb->end( foo => 41 );       # not a second time
$cb->end( food => 'bard' );  # we didn't expect that
# Not fond of dying in async code, so probably 
#   add some customizable error handler,
#   say $cb->on_error(sub {...}); or like that

# Finally, last hanging task is done
$cb->end( baz => 137 );      # BOOM!
# at this point, sub {}->( { foo=>42, bar=>undef, baz=>137 } ) has bee
+n called

Looks easy enough to implement, but maybe there's already something like that?

[salva]

Of course, this can be done. Set up lots of callbacks that share a hash; delete keys from that hash whenever a task finishes; call the megacallback when the hash is empty. I wonder if there's a more civilized way of doing it, maybe some CPAN module?

I find this approach pretty civilized. I am sure any module providing that functionality will actually do it in this same way under the hood.

[Dallaylaen]

Crossposted to stackoverflow and got a reply that satisfied me.

The author, LeoNerd, suggests using either Future or Async::MergePoint which both fullfill my needs - Future in a more generalized way, and MergePoint exactly in the form I imagined when writing my post.