Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

Re^2: Cookie protected web page and file downloading

by reTard (Sexton)
on Jun 02, 2005 at 03:42 UTC ( #462736=note: print w/replies, xml ) Need Help??


in reply to Re: Cookie protected web page and file downloading
in thread Cookie protected web page and file downloading

I've had a look at Mech but I can't see where to specify the HTTP proxy details?
thanks
  • Comment on Re^2: Cookie protected web page and file downloading

Replies are listed 'Best First'.
Re^3: Cookie protected web page and file downloading
by tlm (Prior) on Jun 02, 2005 at 03:49 UTC
      So far I have:
      #!/usr/bin/perl use Data::Dumper; use LWP::UserAgent; use WWW::Mechanize; my $mech = WWW::Mechanize->new(cookie_jar => {}, agent => "WWW-Mechani +ze/0.01"); $url = 'http://www.ibm.com/servers/eserver/support/pseries/aixfixes.ht +ml'; my $ua = new LWP::UserAgent; $ua->proxy('http','192.168.1.248'); $mech->get( $url ); $mech->follow_link( text_regex => qr/More fix services/); $mech->follow_link( text_regex => qr/AIX 5.3/); $mech->follow_link( text_regex => qr/Data file for AIX 5.3/); print Dumper $mech;

      But this fails as it is not going through the proxy
      Thanks

      UPDATED the print dump shows

      'status' => 500, 'content' => '500 Can\'t connect to www.ibm.com:80 (B +ad hostname \'www.ibm.com\')

        I would not expect what you have to work. That's because you have created two user agent objects, $ua and $mech (yes the latter is a user agent object too, because WWW::Mechanize is a subclass of LWP::UserAgent), one ($ua) that is configured for use with a proxy (but is otherwise not used), and the other ($mech) that is not configured to use a proxy. I think what you want is something more like this:

        #!/usr/bin/perl use Data::Dumper; use WWW::Mechanize; my $mech = WWW::Mechanize->new(cookie_jar => {}, agent => "WWW-Mechani +ze/0.01"); $url = 'http://www.ibm.com/servers/eserver/support/pseries/aixfixes.ht +ml'; $mech->proxy('http','192.168.1.248'); $mech->get( $url ); $mech->follow_link( text_regex => qr/More fix services/); $mech->follow_link( text_regex => qr/AIX 5.3/); $mech->follow_link( text_regex => qr/Data file for AIX 5.3/); print Dumper $mech;
        Note that what I have done is treat $mech as an LWP::UserAgent object. (If it's not clear what's going on, take a look at perltoot.)

        BTW, you should get into the habit of checking for the success of requests made through the $mech object; you do this with its is_success method.

        the lowliest monk

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://462736]
help
Chatterbox?
[Corion]: ambrus: Well, for mathematical notation, I find blackboard much more convenient than a computer. But when inserting text or moving text around, the computer wins obviously
[ambrus]: But either of these is a big problem in practice, so I'd need to spend like thirty years of my life to solve (a) and five more years to solve (b)
[ambrus]: Corion: yes, CURRENTLY the blackboard is more convenient
[ambrus]: and it's not like I want to ban blackboards anyway
[ambrus]: you already have blackboards and a canvas for projector or overhead transparencies (or positive film slide projector, not used for maths) canvas in the same lecture halls today, and switch in a few minutes between presentations,
[ambrus]: they're only difficult to use together.
[ambrus]: overhead transparencies are a nice convenience by the way that mix the two slides, because you can write them in advance and edit them during the presentation easily. but they're not very much in fasion these days.
[ambrus]: you can even print them.
[Discipulus]: ambrus i'm trying out MremoteNG which wrap putty and rdp and many other things..
[Corion]: ambrus: Yes, ideally you would have the ease of overhead projection transparencies and pens drawing on them, combined with the computer generated slide text...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (11)
As of 2017-09-26 10:26 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    During the recent solar eclipse, I:









    Results (293 votes). Check out past polls.

    Notices?