http://www.perlmonks.org?node_id=533741

Eyck has asked for the wisdom of the Perl Monks concerning the following question:

Salut,

I've got a little problem with great DAV-explorer - 'dave', it has got a great interface, works fine, until you try to impress your peers and show off it's abilities by transferring securely and easily (https) few gigabyte sized file...
your workstation suddenly melts. Not good.

Here's the culprit:

binmode F; while(<F>) { $content .= $_; } close F;

All is well and good, although this code appears not in dave itself, but in HTTP::Dav library (line 877, /usr/share/perl5/HTTP/DAV.pm), since this problem exists in library and not in dave, it's way harder to fix, especially since the problem here actually is not with HTTP::Dav, but with LWP itself, because it requires $content parameter to be copied scalar..

Fixing LWP would be rather heroic task, it is widely used, you would have to either change it's API, or maybe extend it to support passing filehandles...

What are the options? It's obvious that one would try to make his fixes as non-invasive as possible (fixing relatively esoteric HTTP::Dav) but the more logical and usefull thing would be to fix the problem at it's source.., is there some way to MMAP a file into scalar? how should one go about changing as popular lib as LWP?

Replies are listed 'Best First'.
Re: Fixing a problem in deep object hierarchy (slurp in a library)
by ikegami (Patriarch) on Mar 01, 2006 at 22:09 UTC

    HTTP::Request::Common supports POSTing files without loading them into memory by passing a callback to the content method of the request. The same could be done for a PUT request. I don't see where passing a callback to content is documented, but it is required for documented features. Look at the code in HTTP::Request::Common that wouldn't get executed if $DYNAMIC_FILE_UPLOAD was false.

    The catch is that HTTP::DAV relies on content being a string in numerous places, as far as I can tell. If so, fixing it to use a callback for content would entail a lot of work.

      Still, that would be the way to go, thanks for the info about HTTP::Request:Common.