Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses

Re: Can You Explain How to Check a Link for Deadness

by choocroot (Friar)
on May 12, 2002 at 19:39 UTC ( #166016=note: print w/replies, xml ) Need Help??

in reply to Can You Explain How to Check a Link for Deadness

# The script take the URL list on the command line # So, you should call your script like this : # perl http://site/f1.html http://site/f2.html ... # Command line arguments are stored in the ARGV list. # For each URL in ARGV, place this current URL in $_ # $_ the the "default" variable in Perl. for (@ARGV){ # Remove the leading "http://" part of $_ s|http://||; # Extract the server name in $1 and the file name in $2 # from $_ (see "perlre" documentation for this) m|([^/]+)(.*)|; # Open a tcp socket connection to the server $1 on port 80 my $s=IO::Socket::INET->new( PeerAddr=>$1, PeerPort=>80, Proto=>'tcp +', Type=>SOCK_STREAM ); # Send a simple HTTP GET request to the server for # file $2 or "/" if $2 is not defined. print $s "GET ".($2||'/')." HTTP/1.0\nHost: $1 \n\n"; # Read the first line of the answer (with <$s>) from the # server and print "Link xxx is validated" if the server # answered positively to the request (server answers # "HTTP 200 OK" when file is present) print "Link $_ is validated\n" if <$s>=~/200 OK/; # Close the connection close $s; # and treat the next URL }

You can use the LWP package (launch perl -e 'use LWP' to check if LWP is installed).
With LWP this could be rewritten like this:

use strict; use LWP::UserAgent; my $ua = LWP::UserAgent->new; foreach my $url (@ARGV) { my $request = HTTP::Request->new( GET => "$url" ); my $response = $ua->request( $request ); if( $response->is_success ) { print "link $url is ok\n" } }
LWP provide a higher abstraction, you don't need to handle the "low level" socket creation/communication.
Read the documentation for LWP and HTTP::Request for futher details.

Good luck :)

Replies are listed 'Best First'.
Re: Re: Can You Explain How to Check a Link for Deadness
by belg4mit (Prior) on May 12, 2002 at 19:44 UTC
    A pretty thorough explanation, ++. A tip, perl -MLWP is shorter and will do the same thing.

    perl -pew "s/\b;([mnst])/'$1/g"

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://166016]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (3)
As of 2021-01-25 23:40 GMT
Find Nodes?
    Voting Booth?