Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic

LWP-Download 401 Authorization Required

by Jeri (Scribe)
on Jun 04, 2012 at 19:52 UTC ( #974362=perlquestion: print w/replies, xml ) Need Help??
Jeri has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks. I'm trying to directly download a file from a site that requires authorization. I was able to get the html of the site using LWP::UserAgent, but now I want to download a file directly. I tried using $lwp-download <url> but it too requires authorization. How am I able to directly download a file from a protected site (assuming I have all the information to get onto it)?

  • Comment on LWP-Download 401 Authorization Required

Replies are listed 'Best First'.
Re: LWP-Download 401 Authorization Required
by zentara (Archbishop) on Jun 04, 2012 at 20:03 UTC
    For using LWP see the below example, but you may need WWW::Mechanize for which see Need help using WWW::Mechanize to access proxy and authentication as an example.
    #!/usr/bin/perl use warnings; use strict; use HTTP::Request::Common qw(GET); use LWP::UserAgent; my $url =''; my $filename = substr( $url, rindex( $url, "/" ) + 1 ); #print "$filename\n"; open( IN, ">$filename" ) or die $!; my $user = 'zentara'; my $pass = 'foobar'; my $expected_length; my $bytes_received = 0; my $ua = new LWP::UserAgent; $ua->protocols_allowed( [ 'https'] ); #setup request my $req = GET($url); $req->authorization_basic($user, $pass); #do it my $response = $ua->request($req); if ($response->is_error()) { printf " %s\n", $response->status_line; print "https request error!\n"; } else { my $content = $response->content(); print IN $content; } print $response->status_line; close IN; exit;

    I'm not really a human, but I play one on earth.
    Old Perl Programmer Haiku ................... flash japh
      Hey I am Sam and I really thanks to you your above code snippet helpful to me. thanks, sam
Re: LWP-Download 401 Authorization Required
by pemungkah (Priest) on Jun 05, 2012 at 19:11 UTC
    You may also be running afoul of the fact that LWP doesn't handle cookies itself. It's common that you access a login URL, supplying the username and password via POST; the application returns a cookie, which then must be passed back again when GETting a resource.

    HTTP::Cookies should help. From the synopsis:

    use HTTP::Cookies; $cookie_jar = HTTP::Cookies->new( file => "$ENV{'HOME'}/lwp_cookies.dat", autosave => 1, ); use LWP; my $browser = LWP::UserAgent->new; $browser->cookie_jar($cookie_jar);
    At this point, the getting and saving of cookies should happen pretty much automatically.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://974362]
Approved by ww
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (5)
As of 2018-08-19 09:26 GMT
Find Nodes?
    Voting Booth?
    Asked to put a square peg in a round hole, I would:

    Results (186 votes). Check out past polls.