Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation

LWP and proxy

by Anonymous Monk
on May 08, 2007 at 16:57 UTC ( #614194=perlquestion: print w/replies, xml ) Need Help??

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks, I know this is probably something that probably has been asked quite a few times....I have scanned through a lot of documentation ...a lot of queries posted on various groups etc..But still not able to get through this issue with the proxy on my network. Also the requests are Websensed. Here is a sample script I picked up from the code snippets on perl monks .... but this doesnt work...
#!/usr/bin/perl -w # # pod at tail use strict; use LWP::UserAgent; use Getopt::Long; use Pod::Usage; use URI::URL; use LWP::Debug qw(+); my ($opt_help, $opt_man); GetOptions( 'help!' => \$opt_help, 'man!' => \$opt_man, ) or pod2usage(-verbose => 1) && exit; pod2usage(-verbose => 1) && exit if (defined $opt_help); pod2usage(-verbose => 2) && exit if (defined $opt_man); # Begin config parameters my %parm = ( url => shift, outfile => shift, uatimeout => 120, # seconds before giving up on fetch browser => 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.9) Geck +o/20020310 ', ); my %proxy = ( host => '', # http://host.dom:port id => 'mynw\mylogin', # ntdom\userid pass => 'mypasswd', # empty quotes if no proxy auth ); # End config parameters unless(defined $parm{url}) { print "\n Ooot - you forgot to provide a URL !\n"; Ooot(); } unless(defined $parm{outfile}) { print "\n Ooot - your forgot to provide an outfile !\n"; Ooot(); } print "\n Fetching $parm{url}...\n"; my $ua = new LWP::UserAgent; $ua->agent($parm{browser}); $ua->timeout($parm{uatimeout}); $ua->proxy(http => "$proxy{host}") if (defined $proxy{host}); $parm{url} = new URI::URL($parm{url}); my $req = new HTTP::Request "GET" => ($parm{url}); $req->proxy_authorization_basic ($proxy{id}, $proxy{pass}) if (defined $proxy{id}); my $res = $ua->request($req); if ($res -> is_success) { my $rescont = $res->content; open (OUT, ">$parm{outfile}") or die "Error opening $parm{outfile} for write: $!"; print OUT $rescont; close OUT or die "Error closing $parm{outfile}: $!"; } else { my $resmsg = $res->message; print "Error fetching $parm{url}:\n $resmsg\n\n"; exit; } print " Done! Page saved at '$parm{outfile}'\n\n"; sub Ooot { print "\n --help", "\n --man", "\n", "\n LWP $LWP::VERSION", "\n Perl $]", "\n OS $^O", "\n Program $0", "\n\n", ; exit; } Outputs the following : D:\testperl>perl out.txt Fetching LWP::UserAgent::new: () LWP::UserAgent::proxy: http LWP::UserAgent::request: () LWP::UserAgent::send_request: GET LWP::UserAgent::_need_proxy: Proxied to LWP::Protocol::http::request: () LWP::Protocol::collect: read 623 bytes LWP::Protocol::collect: read 1754 bytes LWP::UserAgent::request: Simple response: Proxy Authentication Require +d Error fetching Proxy Authentication Required ( The ISA Server requires authorizatio +n to fulfi ll the request. Access to the Web Proxy service is denied. )
Please guide me to the right issue behind this.

Replies are listed 'Best First'.
Re: LWP and proxy
by Fletch (Chancellor) on May 08, 2007 at 17:25 UTC

    I've experienced (oh, have I experienced . . . *grumble*) problems with a proxy server running on NT in that it expects to receive NTLM credentials but after an unsuccessful round of expecting that will fallback to accepting Basic authentication instead. That's another thing to get with your proxy administrator about (along with getting them to look in the logs as was previously mentioned).

    And no, I don't have a workaround for the problem in Perl; fortunately in my case there was another proxy that didn't require NTLM available. We did investigate some sort of NTLM proxy (a proxy proxy that provides the real proxy with the NTLM credentials it wants), but I wasn't involved in that part so I can't give you more than that vague pointer.

      Sirs, Can anyone provide more info please? I am still struggling with this...
Re: LWP and proxy
by cengineer (Pilgrim) on May 08, 2007 at 17:13 UTC
    I assume that you are not actually using '' as the actual proxy. Have you checked the log on the proxy server to see if it is receiving the correct username and password combination?
      thats right.. I have just replaced my company's name with mycomp >Have you checked the log on the proxy server to see if it >is receiving the correct username and password >combination? The login and password are my current network login and password. the ones i've typed are the ones with which I am actually logged on to at the moment this is pretty much working....

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://614194]
Approved by kyle
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (1)
As of 2022-01-28 03:41 GMT
Find Nodes?
    Voting Booth?
    In 2022, my preferred method to securely store passwords is:

    Results (72 votes). Check out past polls.