LWP-Download 401 Authorization Required

[Jeri]

Hi Monks. I'm trying to directly download a file from a site that requires authorization. I was able to get the html of the site using LWP::UserAgent, but now I want to download a file directly. I tried using $lwp-download <url> but it too requires authorization. How am I able to directly download a file from a protected site (assuming I have all the information to get onto it)?

[zentara]

For using LWP see the below example, but you may need WWW::Mechanize for which see Need help using WWW::Mechanize to access proxy and authentication as an example.

#!/usr/bin/perl
use warnings;
use strict;
use HTTP::Request::Common qw(GET);
use LWP::UserAgent;

my $url ='https://zentara.zentara.net/~zentara/zentara1.avi';
my $filename = substr( $url, rindex( $url, "/" ) + 1 );
#print "$filename\n";
open( IN, ">$filename" ) or die $!;

my $user = 'zentara';
my $pass = 'foobar';

my $expected_length;
my $bytes_received = 0;

my $ua = new LWP::UserAgent;
$ua->protocols_allowed( [ 'https'] );

 #setup request
 my $req = GET($url); 
 $req->authorization_basic($user, $pass);

#do it         
 my $response = $ua->request($req);

 if ($response->is_error())
            {
                    printf " %s\n", $response->status_line;
                    print "https request error!\n";
            } else {
                    my $content = $response->content();
                    print IN $content;
                }

  print $response->status_line;

close IN;
exit;

[Anonymous Monk]

Hey I am Sam and I really thanks to you your above code snippet helpful to me. thanks, sam

[pemungkah]

You may also be running afoul of the fact that LWP doesn't handle cookies itself. It's common that you access a login URL, supplying the username and password via POST; the application returns a cookie, which then must be passed back again when GETting a resource.

HTTP::Cookies should help. From the synopsis:

  use HTTP::Cookies;
  $cookie_jar = HTTP::Cookies->new(
    file => "$ENV{'HOME'}/lwp_cookies.dat",
    autosave => 1,
  );

  use LWP;
  my $browser = LWP::UserAgent->new;
  $browser->cookie_jar($cookie_jar);

At this point, the getting and saving of cookies should happen pretty much automatically.