Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer

Current download information..

by Anonymous Monk
on Jan 05, 2001 at 00:35 UTC ( #49858=perlquestion: print w/replies, xml ) Need Help??

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I've been writting a program in Perl that downloads files from the net. However, I've found that it would be most helpful to see a "download percentage" or other indication that the download is not only working but how far along it is. The problem is the accual downloading is being handled by one single line of code.
if ($resp->is_success) { open (DOWNLOAD, "> $Filename") || warn; binmode(DOWNLOAD); print DOWNLOAD $resp->content; close (DOWNLOAD) || warn; } ...
How can I break into this to get the information I want? I feel like I'm missing something basic here.. :(( Alucard

Replies are listed 'Best First'.
Re: Current download information..
by Dominus (Parson) on Jan 05, 2001 at 01:34 UTC
    (Summary: The anonymous monk wants a progress bar for his download, which I think might be via HTTP.)

    I suppose you are using LWP::Simple or LWP::UserAgent. You need to use LWP::UserAgent for this. Look in the manual for the following version of the request function:

    $response = $ua->request($request, \&callback, 4096); sub callback { my($data, $response, $protocol) = @_; .... }

    Here callback is a function that gets called automatically every time your program receives 4096 bytes. callback can be responsible for updating the progress status bar.

      That's exactly what I wanted!!! :)) Alucard
      Since you're being incredibly helpful, can I impose on you further than anon_monk did?

      In the callback your @_... so the callback gets all 4096 bytes of $data, the $response is the same as the other $response in the snippet, or is that something else now, and the $protocol (which I assume is "http", "ftp", "file" or whatever else UserAgent handles).

      Once the request is finished, does the original $response contain the full result of the transaction still? And is there a way to insert a last/return clause into the callback which aborts the request?

      Should I just RTFM? {grin}
        Says ichimunki:

        Since you're being incredibly helpful, can I impose on you further than anon_monk did?
        Um, no. Now you're asking me to read you the manual. If you read it and you don't understand, then I will try to answer. You could also try writing a few small example programs to test out your guesses about how it works.

        is there a way to insert a last/return clause into the callback which aborts the request?

        The general pattern for such things in Perl is:

        eval { $ua->request($request, \&callback, 4096) }; if ($@ =~ /^Request aborted/) { # handle aborted request } elsif ($@) { die; } sub callback { # ... die "Request aborted" if ...; # ... }
Re: Current download information..
by btrott (Parson) on Jan 05, 2001 at 01:35 UTC
    Take a look at lwp-download on your machine. You should have it, at least if you have LWP (which it seems like you do). It does something very much like this--gives status on the download progressing--so you could either just use that, or steal some of the code from there.
(fongsaiyuk)Re: Current download information..
by fongsaiyuk (Pilgrim) on Jan 05, 2001 at 01:59 UTC
Re: Current download information..
by jeroenes (Priest) on Jan 05, 2001 at 11:08 UTC
    Also take a look at File copy progress.. Similar problem, some nice solutions.


    I was dreaming of guitarnotes that would irritate an executive kind of guy (FZ)

Re: Current download information..
by Fastolfe (Vicar) on Jan 05, 2001 at 00:54 UTC
    What is $resp? It sounds almost like the download is occurring above the code you pasted, and that $resp is an object holding the results (with the full content of that result). Depending upon the level of abstraction this object is providing you, it may not be possible to do what you're trying to do while still using it. In order to get this kind of progress, you'd need to be inserting progress-checking code directly into the read loop, updating progress information after every block received (or less often). Unless whatever module you're using allows you to hook into that, you'll have to break that out into code you write yourself to retrieve the data. To help further we'd need to know what you're using to retrieve the data.

    Or are you talking about getting progress information while saving $resp->content into the DOWNLOAD filehandle?

      I'm useing LWP::UserAgent, and $resp is the responce from a http GET from my program. Here is a bigger picture of my code..
      $hdrs = new HTTP::Headers(Accept => 'application/octet-stream', User_Agent => 'my program'); $url = new URI::URL($FileURL); ## $FileURL might hold - http://www.wha $req = new HTTP::Request('GET', $url, $hdrs); $ua = new LWP::UserAgent; $ua->timeout($WaitState); ## $WaitState holds a number $resp = $ua->request($req); if ($resp->is_success) { open (DOWNLOAD, "> $Filename") || warn; binmode(DOWNLOAD); print DOWNLOAD $resp->content; close (DOWNLOAD) || warn; return "Success"; } else { return "Fail"; }
      I thought the accual downloading was taking place when "print DOWNLOAD $resp->content;" get's fired off. Does this new code help you understand what I'm attempting to do? Alucard
        Sounds like you got Dominus's advice, and that's the way to go. The download actually happens at this line here:
        $resp = $ua->request($req);
        When you get your $resp object back, the request has already been made and a response (hence 'resp') has been retrieved from the server in its entirety. The content method is just accessing that content retrieved in this step.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://49858]
Approved by root
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (5)
As of 2020-01-23 14:16 GMT
Find Nodes?
    Voting Booth?