Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Re^2: Cookie protected web page and file downloading

by reTard (Sexton)
on Jun 02, 2005 at 03:42 UTC ( #462736=note: print w/ replies, xml ) Need Help??


in reply to Re: Cookie protected web page and file downloading
in thread Cookie protected web page and file downloading

I've had a look at Mech but I can't see where to specify the HTTP proxy details?
thanks


Comment on Re^2: Cookie protected web page and file downloading
Re^3: Cookie protected web page and file downloading
by tlm (Prior) on Jun 02, 2005 at 03:49 UTC
      So far I have:
      #!/usr/bin/perl use Data::Dumper; use LWP::UserAgent; use WWW::Mechanize; my $mech = WWW::Mechanize->new(cookie_jar => {}, agent => "WWW-Mechani +ze/0.01"); $url = 'http://www.ibm.com/servers/eserver/support/pseries/aixfixes.ht +ml'; my $ua = new LWP::UserAgent; $ua->proxy('http','192.168.1.248'); $mech->get( $url ); $mech->follow_link( text_regex => qr/More fix services/); $mech->follow_link( text_regex => qr/AIX 5.3/); $mech->follow_link( text_regex => qr/Data file for AIX 5.3/); print Dumper $mech;

      But this fails as it is not going through the proxy
      Thanks

      UPDATED the print dump shows

      'status' => 500, 'content' => '500 Can\'t connect to www.ibm.com:80 (B +ad hostname \'www.ibm.com\')

        I would not expect what you have to work. That's because you have created two user agent objects, $ua and $mech (yes the latter is a user agent object too, because WWW::Mechanize is a subclass of LWP::UserAgent), one ($ua) that is configured for use with a proxy (but is otherwise not used), and the other ($mech) that is not configured to use a proxy. I think what you want is something more like this:

        #!/usr/bin/perl use Data::Dumper; use WWW::Mechanize; my $mech = WWW::Mechanize->new(cookie_jar => {}, agent => "WWW-Mechani +ze/0.01"); $url = 'http://www.ibm.com/servers/eserver/support/pseries/aixfixes.ht +ml'; $mech->proxy('http','192.168.1.248'); $mech->get( $url ); $mech->follow_link( text_regex => qr/More fix services/); $mech->follow_link( text_regex => qr/AIX 5.3/); $mech->follow_link( text_regex => qr/Data file for AIX 5.3/); print Dumper $mech;
        Note that what I have done is treat $mech as an LWP::UserAgent object. (If it's not clear what's going on, take a look at perltoot.)

        BTW, you should get into the habit of checking for the success of requests made through the $mech object; you do this with its is_success method.

        the lowliest monk

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://462736]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (11)
As of 2014-11-28 08:31 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My preferred Perl binaries come from:














    Results (194 votes), past polls