Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

comment on

( [id://3333]=superdoc: print w/replies, xml ) Need Help??
Exactly, server-side.

Squid is interesting I think, but I don't just want to proxy connections, I will need to do some programming.

One of the problems I have is that the PERL modules for S3 don't stream data, which means that if my application is sending a 2GB file to S3 then that 2GB file gets loaded into memory before it gets sent, that's bad. So I will store files over 500 MB locally and keep that info in a table.

But while I'm doing tables I might as well cache the most used files locally on my server instead of sending users to pick them up from S3 and incurring the bandwidth charge, say cache up to 100GB (the size of the local drive) before rotating out of the cache, and then how to determine which files should and shouldn't be in the cache.

This all seems to me to be something that somebody has to have done before, maybe not with S3, but just storing content and caching it. Maybe not...


In reply to Re^3: File caching for external storage by tosh
in thread File caching for external storage by tosh

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others cooling their heels in the Monastery: (2)
As of 2024-06-22 20:05 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?
    erzuuli‥ 🛈The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.