Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re^2: Determining Content-Length when there is no Content-Length header

by erroneousBollock (Curate)
on Sep 30, 2007 at 03:25 UTC ( #641752=note: print w/replies, xml ) Need Help??


in reply to Re: Determining Content-Length when there is no Content-Length header
in thread Determining Content-Length when there is no Content-Length header

You can try to use a partial GET (byte range) to seek around the file to find its end by trial (and avoid downloading 20GB), but I'm pretty sure 99% of the resources supporting a partial GET will also report a Content-Length in the header.
They're definitely not required to be present together.

Furthermore, the server doesn't necessarily have to tell you that it doesn't support range requests at all (let alone in a useful manner).
The RFC says:

Note: clients cannot depend on servers to send a 416 (Requested range not satisfiable) response instead of a 200 (OK) response for an unsatisfiable Range request-header, since not all servers implement this request-header.
The definitive answer is no.

Now, for hacker...

Alternatively, why not use the ':content_cb' callback of LWP::UserAgent. With that callback, you can implement your own semantics for max-content-length; if you decide you no-longer want to fetch the file when you reach say 2k, just abort the request by die()ing.

-David

  • Comment on Re^2: Determining Content-Length when there is no Content-Length header

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://641752]
help
Chatterbox?
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (3)
As of 2018-07-18 07:09 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    It has been suggested to rename Perl 6 in order to boost its marketing potential. Which name would you prefer?















    Results (383 votes). Check out past polls.

    Notices?