Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

anyevent http 404code

by liuweichuan (Acolyte)
on Oct 19, 2016 at 05:49 UTC ( #1174272=perlquestion: print w/replies, xml ) Need Help??

liuweichuan has asked for the wisdom of the Perl Monks concerning the following question:

why can not open this web page and return 404 code via anyevent::http. but LWP can get this page. then can open other web site page.

use AnyEvent::HTTP; use AnyEvent; my @urls = qw( http://www.zol.com/detail/cell_phone/QiKU/26847001.html?spm=634.42184 ); my $cv = AnyEvent->condvar; for my $url ( @urls ) { $cv->begin; http_get $url, sub { my ($data, $headers) = @_; foreach my $k (keys %$headers){ print $k," : ",$headers->{$k},"\n"; } $cv->end; } } $cv->recv;

Replies are listed 'Best First'.
Re: anyevent http 404code
by Krambambuli (Curate) on Oct 19, 2016 at 10:28 UTC
    The relevant info in the AnyEvent::HTTP documentation refers to request headers and reads
    ---
       You really should provide your own "User-Agent:" header value that is appropriate for your program - I wouldn't be surprised if the default AnyEvent  string gets blocked by webservers sooner or later.
    
    ---
    
    The server blocks the AnyEvent request, but works ok if you indicate another valid User-Agent. Like
    http_get $url, headers => { 'User-Agent' => 'curl/7.47.1' }, sub { [...]
      headers => { 'User-Agent' => 'curl/7.47.1' }, for your code has syntax error
      for other web "https", error:TLS support not available on this system. done. install net ssleay it's ok now
        it can run now. install net ssleay

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1174272]
Approved by Erez
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (4)
As of 2020-10-20 03:53 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My favourite web site is:












    Results (208 votes). Check out past polls.

    Notices?