Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Asynchronous HTTP requests

by perlfan1020 (Initiate)
on Nov 23, 2014 at 05:41 UTC ( #1108139=perlquestion: print w/replies, xml ) Need Help??

perlfan1020 has asked for the wisdom of the Perl Monks concerning the following question:

I need to resolve a large number of HTTP requests asynchronously and write the retrieved HTTP pages to files named from the URLs.

AnyEvent::HTTP seems to be a good choice. However, in the call back as the following: I can't associate the $data to the $url.

http_get $url, key => value..., $cb->($data, $headers)

Any ideas? Thanks in advance!

Replies are listed 'Best First'.
Re: Asynchronous HTTP requests
by Loops (Curate) on Nov 23, 2014 at 06:01 UTC

    It's in the header...

    use AnyEvent::Loop; use AnyEvent::HTTP; http_get 'http://yahoo.com', sub { my ($body, $hdr) = @_; die $hdr->{U +RL} }; AnyEvent::Loop::run;

    And if it wasn't, or there is other information you'd like to use.. capture it as a closure when the sub is defined.. for instance:

    use AnyEvent::Loop; use AnyEvent::HTTP; use feature 'say'; for my $url ('yahoo.com', 'google.com') { http_get "http://$url", sub { say STDERR $url }; } AnyEvent::Loop::run;

    Two separate callback subs are created, and in each of them $url will be set to the value it held when the http_get was run.

Re: Asynchronous HTTP requests
by ikegami (Pope) on Nov 25, 2014 at 19:17 UTC

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1108139]
Approved by Loops
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (3)
As of 2021-05-15 12:36 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Perl 7 will be out ...





    Results (150 votes). Check out past polls.

    Notices?