LWP question

[sri1230]

Can some one tell me why the get in this code fails to retrieve anyhting? #!/usr/bin/perl use strict; use warnings; use LWP::Simple; use HTML::LinkExtor; use Data::Dumper; my $content = get("http://www.yahoo.com/"); #Get web page in content die "get failed" if (!defined $content); my $parser = HTML::LinkExtor->new(); #create LinkExtor object with no callbacks $parser->parse($content); #parse content my @links = $parser->links; #get list of links print Dumper \@links; #print list of links out.

[VinsWorldcom]

As written below, it works for me:

{C} > cat test.pl
#!/usr/bin/perl

use strict;
use warnings;
use LWP::Simple;
use HTML::LinkExtor;
use Data::Dumper;

my $content = get("http://www.yahoo.com/"); #Get web page in content

die "get failed" if (!defined $content);

my $parser = HTML::LinkExtor->new(); #create LinkExtor object with no 
+callbacks
$parser->parse($content);   #parse content
my @links = $parser->links; #get list of links
print Dumper \@links;       #print list of links out.

{C} > perl -c test.pl
test.pl syntax OK

{C} > perl test.pl
$VAR1 = [
          [
            'link',
            'href',
            'http://l.yimg.com/a/lib/arc/core_1.0.5.css'
          ],
          [
            'link',
...OUTPUT TRUNCATED...

[eric256]

I know in the past i've had issues with several of the search engines where they will block requests by IP if they see repeated requests by UserAgents that look like bots/spiders/etc. (Oh and please read that whole section below your post about formatting.

[sri1230]

Thanks a bunch guys. It was a proxy issue, as i was using my work machine which was blocking it.

[fod]

It works for me - I'd say you've got a connection/firewall/proxy problem

output (long):

Read more... (26 kB)