Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

anyevent http 404code

by liuweichuan (Acolyte)
on Oct 19, 2016 at 05:49 UTC ( #1174272=perlquestion: print w/replies, xml ) Need Help??

liuweichuan has asked for the wisdom of the Perl Monks concerning the following question:

why can not open this web page and return 404 code via anyevent::http. but LWP can get this page. then can open other web site page.

use AnyEvent::HTTP; use AnyEvent; my @urls = qw( http://www.zol.com/detail/cell_phone/QiKU/26847001.html?spm=634.42184 ); my $cv = AnyEvent->condvar; for my $url ( @urls ) { $cv->begin; http_get $url, sub { my ($data, $headers) = @_; foreach my $k (keys %$headers){ print $k," : ",$headers->{$k},"\n"; } $cv->end; } } $cv->recv;

Replies are listed 'Best First'.
Re: anyevent http 404code
by Krambambuli (Curate) on Oct 19, 2016 at 10:28 UTC
    The relevant info in the AnyEvent::HTTP documentation refers to request headers and reads
    ---
       You really should provide your own "User-Agent:" header value that is appropriate for your program - I wouldn't be surprised if the default AnyEvent  string gets blocked by webservers sooner or later.
    
    ---
    
    The server blocks the AnyEvent request, but works ok if you indicate another valid User-Agent. Like
    http_get $url, headers => { 'User-Agent' => 'curl/7.47.1' }, sub { [...]
      headers => { 'User-Agent' => 'curl/7.47.1' }, for your code has syntax error
      for other web "https", error:TLS support not available on this system. done. install net ssleay it's ok now
        it can run now. install net ssleay

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1174272]
Approved by Erez
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (4)
As of 2022-12-06 21:32 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?