Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

Re: Determining Content-Length when there is no Content-Length header

by calin (Deacon)
on Sep 30, 2007 at 01:26 UTC ( #641741=note: print w/replies, xml ) Need Help??


in reply to Determining Content-Length when there is no Content-Length header

You can try to use a partial GET (byte range) to seek around the file to find its end by trial (and avoid downloading 20GB), but I'm pretty sure 99% of the resources supporting a partial GET will also report a Content-Length in the header.

Short answer: no, AFAIK

Update:Here's what I found in the standard. I don't quite understand the verbiage, but take a look at this.

  • Comment on Re: Determining Content-Length when there is no Content-Length header

Replies are listed 'Best First'.
Re^2: Determining Content-Length when there is no Content-Length header
by erroneousBollock (Curate) on Sep 30, 2007 at 03:25 UTC
    You can try to use a partial GET (byte range) to seek around the file to find its end by trial (and avoid downloading 20GB), but I'm pretty sure 99% of the resources supporting a partial GET will also report a Content-Length in the header.
    They're definitely not required to be present together.

    Furthermore, the server doesn't necessarily have to tell you that it doesn't support range requests at all (let alone in a useful manner).
    The RFC says:

    Note: clients cannot depend on servers to send a 416 (Requested range not satisfiable) response instead of a 200 (OK) response for an unsatisfiable Range request-header, since not all servers implement this request-header.
    The definitive answer is no.

    Now, for hacker...

    Alternatively, why not use the ':content_cb' callback of LWP::UserAgent. With that callback, you can implement your own semantics for max-content-length; if you decide you no-longer want to fetch the file when you reach say 2k, just abort the request by die()ing.

    -David

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://641741]
help
Chatterbox?
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (6)
As of 2018-01-19 11:43 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    How did you see in the new year?










    Results (217 votes). Check out past polls.

    Notices?