Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask

Download a file from website

by rahulruns (Scribe)
on Sep 13, 2012 at 07:38 UTC ( #993399=perlquestion: print w/replies, xml ) Need Help??
rahulruns has asked for the wisdom of the Perl Monks concerning the following question:

Need to download a file from a website I have tried using LWP::Simple and curl but the connection closes before the entire file is downloaded. The file size is around 20MB and only around 6MB is downloaded

LWP::Simple code use LWP::Simple; my $url = ' +l'; my $file = 'pigtutorial.tar.gz'; my $status = getstore($url, $file); die "Error $status on $url" unless is_success($status); CURL code my $url = `curl -C - -o pigtutorial.tar.gz +nfluence/display/PIG/PigTutorial`;

Replies are listed 'Best First'.
Re: Download a file from website
by frozenwithjoy (Priest) on Sep 13, 2012 at 08:06 UTC
    Does it work if you try to do it outside of Perl? When I run your curl command in terminal (curl -C - -o pigtutorial.tar.gz, I just get a 65kb file that is the html of the site (and not a tarball). Are you pointing to the correct place?

      Yes it works outside PERL. I am able to download the file and extract it

      rightly replied by frozenwith joy. When you check the file downloaded file from the link it has a href for the location of exact file and now it downloads from that location mentioned in href. This thread could be closed

Re: Download a file from website
by Anonymous Monk on Sep 13, 2012 at 07:54 UTC

    If the remote server is timing you out, there is nothing you can do, esp if it doesn't support resuming

    wget -c -O filename -- http...

    sub LWP::UserAgent::resume

      I went through the code provide for LWP::UserAgent but it is taking time to make out where do I need to change the specific URL and filename. Could you help on that part

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://993399]
Approved by davido
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (8)
As of 2017-11-19 16:33 GMT
Find Nodes?
    Voting Booth?
    In order to be able to say "I know Perl", you must have:

    Results (282 votes). Check out past polls.