Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Which process management module to switch from Coro?

by sleet (Sexton)
on Dec 30, 2016 at 19:33 UTC ( #1178673=perlquestion: print w/replies, xml ) Need Help??

sleet has asked for the wisdom of the Perl Monks concerning the following question:

I need to convert a script that tests concurrent submits against webapps. It uses Coro to ensure multiple submits happened at the same time. Since Coro is unusable these days without patching, I want to switch this over to using multiple processes. But I haven't been able to find any process management module that would allow coordinating the children processes like I do now. This is the code snippet I use now with Coro to do it. Is there a module that will let me do similar with processes? I know I could probably use Mojolicious, but that would require making even more fundamental changes than I'm prepared to at this time.
# Make all the form submissions happen at same instant. Wait until # every thread finishes their initial requests and processing before # submitting. use Coro 'async'; use LWP::Protocol::AnyEvent::http; BEGIN { *CORE::GLOBAL::sleep = \&Coro::AnyEvent::sleep; } my @q = map { async { prepare_requests() } } 1 .. $count; @q = map { $_->join } @q; @q = map { async { submit_form(@_) } $_ } @q; $_->join for @q;

Replies are listed 'Best First'.
Re: Which process management module to switch from Coro?
by sleet (Sexton) on Dec 31, 2016 at 19:21 UTC
    I found a solution. forks::shared and MCE::Shared both support conditional variables and broadcast.
Re: Which process management module to switch from Coro?
by genio (Beadle) on Dec 30, 2016 at 19:49 UTC

    Many people find LeoNerd's Net::Async::HTTP and accompanying IO::Async and Future to be very useful.

    Alternately, there's sri's Mojo::UserAgent. The Mojo::IOLoop makes async requests super simple.

    There are, as always, many ways to accomplish this task. Find the one that suits your style best and move along. Hope that helps.
      As I said, I'd rather not make fundamental changes if I don't need to. I want to keep the LWP request processing if possible. If I need to rewrite from scratch, I'd prefer AnyEvent since it's API and resource use is better than Net::Async.
Re: Which process management module to switch from Coro?
by Corion (Pope) on Dec 30, 2016 at 21:12 UTC

    Why not simply use threads instead?

    Also, I think that Coro should work again, at least, I'm using it with 5.20 without patches and I think it works with 5.24+ as well.

      Not quite it seems.

      Coro installs cleanly for 5.20.3 but fails to compile for 5.24.0.

      Threads aren't an option if your perl isn't compiled to support them. And last I checked, they only worked well under Windows. Coro doesn't compile on OS X with 5.24 and you can see on the CPAN Testers page that there are many other failures: http://www.cpantesters.org/distro/C/Coro.html

        If you can't recompile your Perl to use threads, maybe Parallel::ForkManager can help you to parallelize your requests enough?

        I'm using perl 5.20 until Coro is patched to work with later versions.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1178673]
Approved by davies
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (3)
As of 2021-05-07 04:31 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Perl 7 will be out ...





    Results (86 votes). Check out past polls.

    Notices?