Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

WWW::Mechanize - error on geting non-existing page

by Anonymous Monk
on Nov 05, 2011 at 16:15 UTC ( #936163=perlquestion: print w/ replies, xml ) Need Help??
Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hi all
I try to get a few URLs with WWW::Mechanize. This works fine as long as the URL exists. If I hit an non-existing URL the script dies.
How can I get the script to go on with the next URL after the non-existing one?
Thanks for your help

Comment on WWW::Mechanize - error on geting non-existing page
Re: WWW::Mechanize - error on geting non-existing page
by bichonfrise74 (Vicar) on Nov 05, 2011 at 16:47 UTC
    Can you show us what you have done so far?
Re: WWW::Mechanize - error on geting non-existing page
by moritz (Cardinal) on Nov 05, 2011 at 16:51 UTC
Re: WWW::Mechanize - error on geting non-existing page
by Limbic~Region (Chancellor) on Nov 05, 2011 at 18:20 UTC
    Anonymous Monk,
    With current versions of WWW::Mechanize, autocheck is enabled causing $mech to die in these situations. Here is how you might want to handle it:
    eval { $mech->get($url); }; if ($@) { if ($mech->status == 404) { # handle page not found } else { die "Mech failed to fetch '$url': $@"; } }

    Cheers - L~R

      You also might want to switch to WWW::Mechanize::GZip, since some versions of the original WWW::Mechanize seem to announce to handle HTTP compression but don't in reality. This might result in "unparseable" results returned from the webserver.
      Don't use '#ff0000':
      use Acme::AutoColor; my $redcolor = RED();
      All colors subject to change without notice.

        You also might want to switch to WWW::Mechanize::GZip, since some versions of the original WWW::Mechanize seem to announce to handle HTTP compression but don't in reality. This might result in "unparseable" results returned from the webserver.

        I doubt this ever happens :) but if it did, the solution is to always install the latest WWW::Mechanize , LWP, Compress::Zlib

        If you were going to install WWW::Mechanize::GZip, you might install the latest WWW::Mechanize , LWP, Compress::Zlib instead

      Thanks a bunch, problem solved!

      You can also just set onerror => undef which is easier and less error prone than eval/try style code.

        Your Mother,
        I honestly can't think of any reason this might be true. Do you also establish a DBI connection without RaiseError set to true? It is akin to not checking the return code of open. If something goes wrong, I want to know about it very loudly instead of blindly assuming everything is ok.

        Can you give me an example of where setting it to false is better? I assure you I haven't adopted this strategy as a cargo cult practice and am genuinely interested in hearing your views on how this is better.

        Cheers - L~R

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://936163]
Approved by moritz
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (9)
As of 2015-07-03 08:46 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (50 votes), past polls