Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number

Need to test HTTP page responses

by cantfindausername (Novice)
on Oct 22, 2008 at 10:33 UTC ( #718690=perlquestion: print w/ replies, xml ) Need Help??
cantfindausername has asked for the wisdom of the Perl Monks concerning the following question:

Hi, first time posting on PerlMonks, but have found plenty of information here in the past.

I need to write a script that will take an array or URLs (http redirects) and test wether or not they are still valid - what the response code is.

I have seen that there are a few modules that look to be useful around this area, but I really am lost with where to start on this one.

Does anyone have any idea what I could use?

Thanks in advance.

Comment on Need to test HTTP page responses
Replies are listed 'Best First'.
Re: Need to test HTTP page responses
by b10m (Vicar) on Oct 22, 2008 at 12:21 UTC

    As stated, LWP::UserAgent will be of help here, but make sure to look at the requests_redirectable option if you want to actually see if the URL returns a 300-code. If you leave HEAD and GET as requests_redirectable values, you'll get the results of the codes after the redirection.

    use LWP::UserAgent; my $ua = LWP::UserAgent->new( requests_redirectable => [] ); while(<DATA>) { chomp; my $r = $ua->head($_); print $_,": ",$r->code,"\n"; } __DATA__

    In this case, the tinyurl URL will return 301. If you leave the request_redirectable option as default, you'll get 200 as status code (coming from, not

Re: Need to test HTTP page responses
by lamp (Chaplain) on Oct 22, 2008 at 10:36 UTC
      Thanks for the reply.

      I've tried using the example in the link, and all I'm getting is: 500 Can't locate object method "format_request" via package "LWP::Protocol::http::Socket"

      Any ideas what that means?

        Do hou have Net::HTTP installed ?

Re: Need to test HTTP page responses
by jonadab (Parson) on Oct 22, 2008 at 11:44 UTC
Re: Need to test HTTP page responses
by sugarboy (Beadle) on Oct 22, 2008 at 12:50 UTC
    The required test can be easily achieved by using Test::WWW::Mechanize with Test::More, where it accesses the URL and it Checks for the response codes being returned.

    Example Code looks like:

    use Test::More tests => 1;
    use Test::WWW::Mechanize

    The response it return will be as follows:
    ok - Got <<URL>> ok

      Thanks all for your help so far.

      I've added the LWP::Debug to the script and for some reason every request is coming through as

      LWP::UserAgent::new: ()
      LWP::UserAgent::request: ()
      LWP::UserAgent::send_request: HEAD http://bonnard/
      LWP::UserAgent::_need_proxy: Not proxied
      LWP::Protocol::http::request: ()
      LWP::UserAgent::request: Simple response: Internal Server Error
      http://bonnard/ 500

      SO need to look into Proxies now me thinks...

Re: Need to test HTTP page responses
by dorward (Curate) on Oct 22, 2008 at 18:01 UTC
    checklink is decent - it expects input in the form of an HTML document though. However, you could either hack into it to take an array, or generate an HTML document from your array.
      I don't even know what that means. :-(

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://718690]
Approved by Corion
Front-paged by Corion
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (6)
As of 2015-11-27 23:28 GMT
Find Nodes?
    Voting Booth?

    What would be the most significant thing to happen if a rope (or wire) tied the Earth and the Moon together?

    Results (734 votes), past polls