Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

Re^2: Help, AnyEvent::HTTP no work!

by ikegami (Patriarch)
on May 09, 2016 at 16:19 UTC ( [id://1162559]=note: print w/replies, xml ) Need Help??


in reply to Re: Help, AnyEvent::HTTP no work!
in thread Help, AnyEvent::HTTP no work!

Mixing Coro and AnyEvent makes very little sense in your case.

I have no idea why you say that.

Making a fixed number of HTTP requests in parallel:

use AE qw( ); use AnyEvent::HTTP qw( http_get ); use Coro qw( async ); use Coro::Channel qw( ); sub make_request { my ($url, $cb) = @_; http_get($url, ..., sub { ... $cb->(); }, ); } my $num_workers = 10; my $q = Coro::Channel->new(); my @threads; for (1..$num_workers) { push @threads, async { while (my $url = $q->get()) { my $cv = AE::cv(); make_request($url, $cv); $cv->recv(); } } } while (my $url = get_next_url()) { $q->put($url); } $q->shutdown(); $_->join() for @threads;

Sure, you can do it without Coro too.

use AE qw( ); use AnyEvent::HTTP qw( http_get ); sub make_request { my ($url, $cb) = @_; http_get($url, ..., sub { ... $cb->(); }, ); } my $max_requests = 10; my $active_requests = 0; my $cv = AE::cv(); while (my $url = get_next_url()) { while ($active_requests == $max_requests) { $cv->recv(); $cv = AE::cv(); } ++$active_requests; make_request($url, sub { --$active_requests; $cv->send(); }); } while ($active_requests) { $cv->recv(); $cv = AE::cv(); }

Whatever floats your boat.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1162559]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others musing on the Monastery: (4)
As of 2024-04-23 15:19 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found