Current download information..

[Anonymous Monk]

I've been writting a program in Perl that downloads files from the net. However, I've found that it would be most helpful to see a "download percentage" or other indication that the download is not only working but how far along it is. The problem is the accual downloading is being handled by one single line of code.

if ($resp->is_success) {
    open (DOWNLOAD, "> $Filename") || warn;
    binmode(DOWNLOAD);
    print DOWNLOAD $resp->content;
    close (DOWNLOAD) || warn;
} ...

How can I break into this to get the information I want? I feel like I'm missing something basic here.. :(( Alucard

[Dominus]

(Summary: The anonymous monk wants a progress bar for his download, which I think might be via HTTP.)

I suppose you are using LWP::Simple or LWP::UserAgent. You need to use LWP::UserAgent for this. Look in the manual for the following version of the request function:

 $response = $ua->request($request, \&callback, 4096);

 sub callback { my($data, $response, $protocol) = @_; .... }

Here callback is a function that gets called automatically every time your program receives 4096 bytes. callback can be responsible for updating the progress status bar.

[Anonymous Monk]

That's exactly what I wanted!!! :)) Alucard

[ichimunki]

Since you're being incredibly helpful, can I impose on you further than anon_monk did?

In the callback your @_... so the callback gets all 4096 bytes of $data, the $response is the same as the other $response in the snippet, or is that something else now, and the $protocol (which I assume is "http", "ftp", "file" or whatever else UserAgent handles).

Once the request is finished, does the original $response contain the full result of the transaction still? And is there a way to insert a last/return clause into the callback which aborts the request?

Should I just RTFM? {grin}

[Dominus]

Says ichimunki:

Since you're being incredibly helpful, can I impose on you further than anon_monk did?

Um, no. Now you're asking me to read you the manual. If you read it and you don't understand, then I will try to answer. You could also try writing a few small example programs to test out your guesses about how it works.

is there a way to insert a last/return clause into the callback which aborts the request?

The general pattern for such things in Perl is:

    eval { $ua->request($request, \&callback, 4096) };
    if ($@ =~ /^Request aborted/) {
      # handle aborted request
    } elsif ($@) {
      die;
    }
    
    sub callback {
      # ...
      die "Request aborted" if ...;
      # ...
    }

[btrott]

Take a look at lwp-download on your machine. You should have it, at least if you have LWP (which it seems like you do). It does something very much like this--gives status on the download progressing--so you could either just use that, or steal some of the code from there.

[fongsaiyuk]

FMTYRWTK, since I really love topics of this sort, check out the following node. Now that you've the capability to receive an event/callback, you'll want something fun to display.

check out this node:A whirleygig for a progress indicator for scripts

fongsaiyuk

[jeroenes]

Also take a look at File copy progress.. Similar problem, some nice solutions.

Cheers,

Jeroen
I was dreaming of guitarnotes that would irritate an executive kind of guy (FZ)

[Fastolfe]

What is $resp? It sounds almost like the download is occurring above the code you pasted, and that $resp is an object holding the results (with the full content of that result). Depending upon the level of abstraction this object is providing you, it may not be possible to do what you're trying to do while still using it. In order to get this kind of progress, you'd need to be inserting progress-checking code directly into the read loop, updating progress information after every block received (or less often). Unless whatever module you're using allows you to hook into that, you'll have to break that out into code you write yourself to retrieve the data. To help further we'd need to know what you're using to retrieve the data.

Or are you talking about getting progress information while saving $resp->content into the DOWNLOAD filehandle?

[Anonymous Monk]

I'm useing LWP::UserAgent, and $resp is the responce from a http GET from my program. Here is a bigger picture of my code..

$hdrs = new HTTP::Headers(Accept => 'application/octet-stream',
User_Agent => 'my program');
$url = new URI::URL($FileURL); ## $FileURL might hold - http://www.wha
+tever.com/me.jpg
$req = new HTTP::Request('GET', $url, $hdrs);
$ua = new LWP::UserAgent;
$ua->timeout($WaitState); ## $WaitState holds a number
$resp = $ua->request($req);
            
if ($resp->is_success) {
    open (DOWNLOAD, "> $Filename") || warn;
    binmode(DOWNLOAD);
    print DOWNLOAD $resp->content;
    close (DOWNLOAD) || warn;
    return "Success";
} else {
    return "Fail";
}

I thought the accual downloading was taking place when "print DOWNLOAD $resp->content;" get's fired off. Does this new code help you understand what I'm attempting to do? Alucard

[Fastolfe]

Sounds like you got Dominus's advice, and that's the way to go. The download actually happens at this line here:

$resp = $ua->request($req);

When you get your $resp object back, the request has already been made and a response (hence 'resp') has been retrieved from the server in its entirety. The content method is just accessing that content retrieved in this step.