Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

WWW::Mechanize - error on geting non-existing page

by Anonymous Monk
on Nov 05, 2011 at 16:15 UTC ( #936163=perlquestion: print w/ replies, xml ) Need Help??
Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hi all
I try to get a few URLs with WWW::Mechanize. This works fine as long as the URL exists. If I hit an non-existing URL the script dies.
How can I get the script to go on with the next URL after the non-existing one?
Thanks for your help

Comment on WWW::Mechanize - error on geting non-existing page
Re: WWW::Mechanize - error on geting non-existing page
by bichonfrise74 (Vicar) on Nov 05, 2011 at 16:47 UTC
    Can you show us what you have done so far?
Re: WWW::Mechanize - error on geting non-existing page
by moritz (Cardinal) on Nov 05, 2011 at 16:51 UTC
Re: WWW::Mechanize - error on geting non-existing page
by Limbic~Region (Chancellor) on Nov 05, 2011 at 18:20 UTC
    Anonymous Monk,
    With current versions of WWW::Mechanize, autocheck is enabled causing $mech to die in these situations. Here is how you might want to handle it:
    eval { $mech->get($url); }; if ($@) { if ($mech->status == 404) { # handle page not found } else { die "Mech failed to fetch '$url': $@"; } }

    Cheers - L~R

      You also might want to switch to WWW::Mechanize::GZip, since some versions of the original WWW::Mechanize seem to announce to handle HTTP compression but don't in reality. This might result in "unparseable" results returned from the webserver.
      Don't use '#ff0000':
      use Acme::AutoColor; my $redcolor = RED();
      All colors subject to change without notice.

        You also might want to switch to WWW::Mechanize::GZip, since some versions of the original WWW::Mechanize seem to announce to handle HTTP compression but don't in reality. This might result in "unparseable" results returned from the webserver.

        I doubt this ever happens :) but if it did, the solution is to always install the latest WWW::Mechanize , LWP, Compress::Zlib

        If you were going to install WWW::Mechanize::GZip, you might install the latest WWW::Mechanize , LWP, Compress::Zlib instead

      Thanks a bunch, problem solved!

      You can also just set onerror => undef which is easier and less error prone than eval/try style code.

        Your Mother,
        I honestly can't think of any reason this might be true. Do you also establish a DBI connection without RaiseError set to true? It is akin to not checking the return code of open. If something goes wrong, I want to know about it very loudly instead of blindly assuming everything is ok.

        Can you give me an example of where setting it to false is better? I assure you I haven't adopted this strategy as a cargo cult practice and am genuinely interested in hearing your views on how this is better.

        Cheers - L~R

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://936163]
Approved by moritz
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (13)
As of 2014-07-29 16:00 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (220 votes), past polls