Re^3: Hobo with a bit of recursion

[marioroy]

Update: Added demonstration using MCE::Inbox

Greetings Veltro,

* I successfully managed to destroyed the original hash $h because I didn't realize the consequence of the nested reference...

Nested structure is moved to the shared manager process where the shared data resides. Use clone to not alter the original hash.

use Clone 'clone' ;

my $h = { ... } ;

my @dupkeys = qw{ nested1 } ;
tie my %result, 'MCE::Shared' ;
@result{ @dupkeys } = @{ clone($h) }{ @dupkeys } ;
print Dumper( $h ) ;

* MCE::Hobo->pending returns 0 when called inside a async block. Is there another way to get the number of pending threads?

MCE::Hobo supports nested spawning. The worker hasn't spawned any Hobo's. A way is via messaging between workers and the manager process. See example on Github using MCE::Inbox (not yet released on MetaCPAN).

use strict;
use warnings;

use MCE::Hobo;
use MCE::Inbox;

##
# MCE::Inbox is not yet released on MetaCPAN. It can be used with any
# parallel module of choice to include threads.
#
# Specify mailbox names during construction. Behind the scene,
# makes a MCE::Shared->queue object per mailbox.
##

my $inbox = MCE::Inbox->new('mngrbox', 'donebox');

MCE::Hobo->init( posix_exit => 1 );

# Workers

mce_async {
    sleep 5;
    $inbox->send('mngrbox', { ident => 'id1', key => 'value1' });
    # do other work, perhaps more send('mngrbox', ...)
    $inbox->send('donebox', 1);
};
mce_async {
    sleep 1;
    $inbox->send('mngrbox', { ident => 'id2', key => 'value2' });
    # do other work, perhaps another send('mngrbox', ...)
    $inbox->send('donebox', 1);
};
mce_async {
    $inbox->send('mngrbox', { ident => 'id3', key => 'value3' });
    # do other work, perhaps send('mngrbox', ...) in a loop
    $inbox->send('donebox', 1);
};

# Manager

while ( my $ret = $inbox->recv('mngrbox') ) {
    printf "ident: %s, key: %s\n", $ret->{ident}, $ret->{key};
    MCE::Hobo->waitone if $inbox->recv_nb('donebox'); # non blocking

    last unless MCE::Hobo->pending;
}

* Another thing I was wondering about, if it is possible to create an hash/object and transfer the control to the mother process (without using the sharing mechanism).

The join method in MCE::Hobo is wantarray-aware. It is useful for obtaining data from the hobo process. The data structure is serialized automatically in memory via MCE::Shared.

use strict;
use warnings;
use MCE::Hobo;

MCE::Hobo->init( posix_exit => 1 );

my @hobos;

push @hobos, mce_async {
    sleep 5;
    return { ident => 1, key => 'value1' };
};
push @hobos, MCE::Hobo->create( sub {
    sleep 1;
    return { ident => 2, key => 'value2' };
});
push @hobos, MCE::Hobo->create( sub {
    my ($ident) = @_;
    return { ident => $ident, key => 'value3' };
}, 3 );

# MCE::Hobo->waitall;

while ( MCE::Hobo->pending ) {
    my $ret = MCE::Hobo->waitone->join;
    print "ident: ", $ret->{ident}, ", key: ", $ret->{key}, "\n";
}

Another way is the on_finish callback which runs under the mother process. It resembles the on_finish callback in Parallel::ForkManager.

use strict;
use warnings;
use MCE::Hobo;

MCE::Hobo->init(
    posix_exit => 1,
    on_finish => sub {
        my ( $pid, $exit, $ident, $signal, $error, $ret ) = @_;
        print "ident: $ident, key: ", $ret->{key}, "\n";
    }
);

MCE::Hobo->create('id1', sub {
    sleep 5;
    return { key => 'value1' };
});
MCE::Hobo->create('id2', sub {
    sleep 1;
    return { key => 'value2' };
});
MCE::Hobo->create('id3', sub {
    return { key => 'value3' };
});

# MCE::Hobo->waitall;

while ( MCE::Hobo->pending ) {
    MCE::Hobo->waitone;
}

Regards, Mario