Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?

Need to test HTTP page responses

by cantfindausername (Novice)
on Oct 22, 2008 at 10:33 UTC ( #718690=perlquestion: print w/replies, xml ) Need Help??
cantfindausername has asked for the wisdom of the Perl Monks concerning the following question:

Hi, first time posting on PerlMonks, but have found plenty of information here in the past.

I need to write a script that will take an array or URLs (http redirects) and test wether or not they are still valid - what the response code is.

I have seen that there are a few modules that look to be useful around this area, but I really am lost with where to start on this one.

Does anyone have any idea what I could use?

Thanks in advance.

Replies are listed 'Best First'.
Re: Need to test HTTP page responses
by b10m (Vicar) on Oct 22, 2008 at 12:21 UTC

    As stated, LWP::UserAgent will be of help here, but make sure to look at the requests_redirectable option if you want to actually see if the URL returns a 300-code. If you leave HEAD and GET as requests_redirectable values, you'll get the results of the codes after the redirection.

    use LWP::UserAgent; my $ua = LWP::UserAgent->new( requests_redirectable => [] ); while(<DATA>) { chomp; my $r = $ua->head($_); print $_,": ",$r->code,"\n"; } __DATA__

    In this case, the tinyurl URL will return 301. If you leave the request_redirectable option as default, you'll get 200 as status code (coming from, not

Re: Need to test HTTP page responses
by lamp (Chaplain) on Oct 22, 2008 at 10:36 UTC
      Thanks for the reply.

      I've tried using the example in the link, and all I'm getting is: 500 Can't locate object method "format_request" via package "LWP::Protocol::http::Socket"

      Any ideas what that means?

        Do hou have Net::HTTP installed ?

Re: Need to test HTTP page responses
by jonadab (Parson) on Oct 22, 2008 at 11:44 UTC
Re: Need to test HTTP page responses
by sugarboy (Beadle) on Oct 22, 2008 at 12:50 UTC
    The required test can be easily achieved by using Test::WWW::Mechanize with Test::More, where it accesses the URL and it Checks for the response codes being returned.

    Example Code looks like:

    use Test::More tests => 1;
    use Test::WWW::Mechanize

    The response it return will be as follows:
    ok - Got <<URL>> ok

      Thanks all for your help so far.

      I've added the LWP::Debug to the script and for some reason every request is coming through as

      LWP::UserAgent::new: ()
      LWP::UserAgent::request: ()
      LWP::UserAgent::send_request: HEAD http://bonnard/
      LWP::UserAgent::_need_proxy: Not proxied
      LWP::Protocol::http::request: ()
      LWP::UserAgent::request: Simple response: Internal Server Error
      http://bonnard/ 500

      SO need to look into Proxies now me thinks...

Re: Need to test HTTP page responses
by dorward (Curate) on Oct 22, 2008 at 18:01 UTC
    checklink is decent - it expects input in the form of an HTML document though. However, you could either hack into it to take an array, or generate an HTML document from your array.
      I don't even know what that means. :-(

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://718690]
Approved by Corion
Front-paged by Corion
LanX probably in my stomach
erix considers showing this to choroba , thinks better of it
[choroba]: LanX we used to mock Hare Krishna processions signing "Hare Rama" carrying the margarine conatiners
[choroba]: :-D

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (10)
As of 2017-11-22 17:01 GMT
Find Nodes?
    Voting Booth?
    In order to be able to say "I know Perl", you must have:

    Results (327 votes). Check out past polls.