Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Need to test HTTP page responses

by cantfindausername (Novice)
on Oct 22, 2008 at 10:33 UTC ( #718690=perlquestion: print w/ replies, xml ) Need Help??
cantfindausername has asked for the wisdom of the Perl Monks concerning the following question:

Hi, first time posting on PerlMonks, but have found plenty of information here in the past.

I need to write a script that will take an array or URLs (http redirects) and test wether or not they are still valid - what the response code is.

I have seen that there are a few modules that look to be useful around this area, but I really am lost with where to start on this one.

Does anyone have any idea what I could use?

Thanks in advance.
Anthony

Comment on Need to test HTTP page responses
Re: Need to test HTTP page responses
by lamp (Chaplain) on Oct 22, 2008 at 10:36 UTC
      Thanks for the reply.

      I've tried using the example in the link, and all I'm getting is: 500 Can't locate object method "format_request" via package "LWP::Protocol::http::Socket"

      Any ideas what that means?

      Thanks,
      Ant
        Do hou have Net::HTTP installed ?


        Krambambuli
        ---
Re: Need to test HTTP page responses
by jonadab (Parson) on Oct 22, 2008 at 11:44 UTC
Re: Need to test HTTP page responses
by b10m (Vicar) on Oct 22, 2008 at 12:21 UTC

    As stated, LWP::UserAgent will be of help here, but make sure to look at the requests_redirectable option if you want to actually see if the URL returns a 300-code. If you leave HEAD and GET as requests_redirectable values, you'll get the results of the codes after the redirection.

    use LWP::UserAgent; my $ua = LWP::UserAgent->new( requests_redirectable => [] ); while(<DATA>) { chomp; my $r = $ua->head($_); print $_,": ",$r->code,"\n"; } __DATA__ http://www.perlmonks.org/ http://tinyurl.com/5j5l5

    In this case, the tinyurl URL will return 301. If you leave the request_redirectable option as default, you'll get 200 as status code (coming from perlmonks.org, not tinyurl.com).

    --
    b10m
Re: Need to test HTTP page responses
by sugarboy (Beadle) on Oct 22, 2008 at 12:50 UTC
    The required test can be easily achieved by using Test::WWW::Mechanize with Test::More, where it accesses the URL and it Checks for the response codes being returned.

    Example Code looks like:

    use Test::More tests => 1;
    use Test::WWW::Mechanize
    $mech->Test::WWW::Mechanize->new;
    $mech=get_ok(URL);

    The response it return will be as follows:
    ok - Got <<URL>> ok

    Thanks,
    Sagar
      Thanks all for your help so far.

      I've added the LWP::Debug to the script and for some reason every request is coming through as

      LWP::UserAgent::new: ()
      LWP::UserAgent::request: ()
      LWP::UserAgent::send_request: HEAD http://bonnard/home.do/
      LWP::UserAgent::_need_proxy: Not proxied
      LWP::Protocol::http::request: ()
      LWP::UserAgent::request: Simple response: Internal Server Error
      http://bonnard/home.do/: 500

      SO need to look into Proxies now me thinks...

      Ant
Re: Need to test HTTP page responses
by dorward (Curate) on Oct 22, 2008 at 18:01 UTC
    checklink is decent - it expects input in the form of an HTML document though. However, you could either hack into it to take an array, or generate an HTML document from your array.
      I don't even know what that means. :-(

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://718690]
Approved by Corion
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (4)
As of 2014-09-18 03:25 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (105 votes), past polls