Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical

Re^5: Cookie protected web page and file downloading

by tlm (Prior)
on Jun 02, 2005 at 05:58 UTC ( #462769=note: print w/ replies, xml ) Need Help??

in reply to Re^4: Cookie protected web page and file downloading
in thread Cookie protected web page and file downloading

I would not expect what you have to work. That's because you have created two user agent objects, $ua and $mech (yes the latter is a user agent object too, because WWW::Mechanize is a subclass of LWP::UserAgent), one ($ua) that is configured for use with a proxy (but is otherwise not used), and the other ($mech) that is not configured to use a proxy. I think what you want is something more like this:

#!/usr/bin/perl use Data::Dumper; use WWW::Mechanize; my $mech = WWW::Mechanize->new(cookie_jar => {}, agent => "WWW-Mechani +ze/0.01"); $url = ' +ml'; $mech->proxy('http',''); $mech->get( $url ); $mech->follow_link( text_regex => qr/More fix services/); $mech->follow_link( text_regex => qr/AIX 5.3/); $mech->follow_link( text_regex => qr/Data file for AIX 5.3/); print Dumper $mech;
Note that what I have done is treat $mech as an LWP::UserAgent object. (If it's not clear what's going on, take a look at perltoot.)

BTW, you should get into the habit of checking for the success of requests made through the $mech object; you do this with its is_success method.

the lowliest monk

Comment on Re^5: Cookie protected web page and file downloading
Download Code
Replies are listed 'Best First'.
Re^6: Cookie protected web page and file downloading
by Corion (Pope) on Jun 02, 2005 at 06:21 UTC

    Instead of manually checking the success/failure after each step, I found it more convenient to have the Mechanize object die on error. This is very convenient for quick development, later, if you want a robust program, you should disable that feature again.

    ... my $mech = WWW::Mechanize->new( autocheck => 1 ); # cookie jar and user agent are set implicitly

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://462769]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (8)
As of 2015-12-01 00:15 GMT
Find Nodes?
    Voting Booth?

    What would be the most significant thing to happen if a rope (or wire) tied the Earth and the Moon together?

    Results (788 votes), past polls