Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Re: Detect Broken links

by CountZero (Bishop)
on Oct 15, 2009 at 09:56 UTC ( #801324=note: print w/replies, xml ) Need Help??


in reply to Detect Broken links

Using LWP::Simple and HTML::SimpleLinkExtor it doesn't have to be more complicated than:
use strict; use warnings; use LWP::Simple; use HTML::SimpleLinkExtor; my $extor = HTML::SimpleLinkExtor->new('http://www.perlmonks.org/'); $extor->parse_url('http://www.perlmonks.org'); print "Checking $_: ", ( head($_) ? 'Good link' : 'Bad link' ), "\n" for $extor->links
Do read the docs for HTML::SimpleLinkExtor as it has a lot more functionality, for instance what kind of links you want to extract.

CountZero

A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

Replies are listed 'Best First'.
Re^2: Detect Broken links
by vishi83 (Pilgrim) on Oct 15, 2009 at 16:29 UTC
    Hi,
    Using HTML::SimpleLinkExtor, can i get the http status of the url hit ?

    Thanks,
    Vishy

    A perl Script without 'strict' is like a House without Roof; Both are not Safe;
      HTML::SimpleLinkExtor only gives you a list of the links in the webpage. To check if you can connect with the resource pointed to by the link I used the head function from LWP::Simple. In scalar context it returns TRUE if successful.

      If you need more detailed information you have to use LWP::UserAgent:

      use strict; use warnings; use LWP::UserAgent; use HTML::SimpleLinkExtor; my $agent = LWP::UserAgent->new; my $extor = HTML::SimpleLinkExtor->new('http://www.perlmonks.org/'); $extor->parse_url('http://www.perlmonks.org'); print "Checking $_: ", $agent->head($_)->code, "\n" for $extor->links
      This will give you the HTTP status code. If you replace ->code by ->message, you get a human readable message instead of the three digit code.

      CountZero

      A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://801324]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (3)
As of 2022-12-07 01:00 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?