Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

Re: LWP::Simple Getstore memory

by vsespb (Hermit)
on Jun 05, 2013 at 06:21 UTC ( #1037136=note: print w/ replies, xml ) Need Help??


in reply to LWP::Simple Getstore memory

LWP::UserAgent can download any huge files into memory. However I did not understand your problem. Could you please provide PoC code for:

crashing the computer and causing the universe to explode
line on a new file on the local machine, but that would cause issues with encoding


Comment on Re: LWP::Simple Getstore memory
Re^2: LWP::Simple Getstore memory
by vsespb (Hermit) on Jun 05, 2013 at 07:28 UTC
    Or maybe you want to download files directly to disk, without huge memory buffer.

    LWP::UserAgent content_file or content_cb will help then
      Yes, I want to download files directly to the disk, without a huge memory buffer. Content_file looks like it practically replaces the need for getstore, as you can simply set the content_file to the filename and store the file that way - without ever reading the file to memory. Am I missing something here, or is it essentially a getstore replacement?
        Yes, replacement.

        You were using LWP::Simple, it's simple interface to LWP::UserAgent.

        But LWP::UserAgent (which have content_file option) is not "Simple" it's full version.

        Also I believe this comment Re: LWP::Simple Getstore memory probably correct, and getstore should not eat so much memory. It looks like it's implemented same way as content_file option.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1037136]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (7)
As of 2014-12-21 11:34 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (104 votes), past polls