Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Need to test HTTP page responses

by cantfindausername (Novice)
on Oct 22, 2008 at 10:33 UTC ( #718690=perlquestion: print w/ replies, xml ) Need Help??
cantfindausername has asked for the wisdom of the Perl Monks concerning the following question:

Hi, first time posting on PerlMonks, but have found plenty of information here in the past.

I need to write a script that will take an array or URLs (http redirects) and test wether or not they are still valid - what the response code is.

I have seen that there are a few modules that look to be useful around this area, but I really am lost with where to start on this one.

Does anyone have any idea what I could use?

Thanks in advance.
Anthony

Comment on Need to test HTTP page responses
Re: Need to test HTTP page responses
by lamp (Chaplain) on Oct 22, 2008 at 10:36 UTC
      Thanks for the reply.

      I've tried using the example in the link, and all I'm getting is: 500 Can't locate object method "format_request" via package "LWP::Protocol::http::Socket"

      Any ideas what that means?

      Thanks,
      Ant
        Do hou have Net::HTTP installed ?


        Krambambuli
        ---
Re: Need to test HTTP page responses
by jonadab (Parson) on Oct 22, 2008 at 11:44 UTC
Re: Need to test HTTP page responses
by b10m (Vicar) on Oct 22, 2008 at 12:21 UTC

    As stated, LWP::UserAgent will be of help here, but make sure to look at the requests_redirectable option if you want to actually see if the URL returns a 300-code. If you leave HEAD and GET as requests_redirectable values, you'll get the results of the codes after the redirection.

    use LWP::UserAgent; my $ua = LWP::UserAgent->new( requests_redirectable => [] ); while(<DATA>) { chomp; my $r = $ua->head($_); print $_,": ",$r->code,"\n"; } __DATA__ http://www.perlmonks.org/ http://tinyurl.com/5j5l5

    In this case, the tinyurl URL will return 301. If you leave the request_redirectable option as default, you'll get 200 as status code (coming from perlmonks.org, not tinyurl.com).

    --
    b10m
Re: Need to test HTTP page responses
by sugarboy (Beadle) on Oct 22, 2008 at 12:50 UTC
    The required test can be easily achieved by using Test::WWW::Mechanize with Test::More, where it accesses the URL and it Checks for the response codes being returned.

    Example Code looks like:

    use Test::More tests => 1;
    use Test::WWW::Mechanize
    $mech->Test::WWW::Mechanize->new;
    $mech=get_ok(URL);

    The response it return will be as follows:
    ok - Got <<URL>> ok

    Thanks,
    Sagar
      Thanks all for your help so far.

      I've added the LWP::Debug to the script and for some reason every request is coming through as

      LWP::UserAgent::new: ()
      LWP::UserAgent::request: ()
      LWP::UserAgent::send_request: HEAD http://bonnard/home.do/
      LWP::UserAgent::_need_proxy: Not proxied
      LWP::Protocol::http::request: ()
      LWP::UserAgent::request: Simple response: Internal Server Error
      http://bonnard/home.do/: 500

      SO need to look into Proxies now me thinks...

      Ant
Re: Need to test HTTP page responses
by dorward (Curate) on Oct 22, 2008 at 18:01 UTC
    checklink is decent - it expects input in the form of an HTML document though. However, you could either hack into it to take an array, or generate an HTML document from your array.
      I don't even know what that means. :-(

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://718690]
Approved by Corion
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (6)
As of 2014-12-20 22:00 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (99 votes), past polls