Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Creating static pages from DB on schedule

by msquare (Initiate)
on Nov 24, 2012 at 10:32 UTC ( #1005360=perlquestion: print w/ replies, xml ) Need Help??
msquare has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks,

I'm not completely sure i'm describing this problem correctly, as it's unlikely this hasn't been done somewhere already, but google-fu has failed to help. I ask for your patience in the reading as it's probably a case of "why not use X?".

Situation is as follows:

  1. I have a large (>1m records, between 20-300 linked attributes per record) data set in place through a somewhat slapdash combination of TT and DBIx::Class:Schema. This works, but with the quantity of users accessing it, it's beginning to show cracks.
  2. The major feature of this platform is Multimedia also known as Really Big Files. The pages around it are really just window dressing, but window dressing that will change just frequently enough to need ease of adjustment.
  3. I have enough familiarity with DNS to make the Big File Hosting seperate and balanced through a bunch of seperate servers, but I do not know enough about distributed databases to make them work with the resources at hand.

Thus what I would like to make is a collection of static database pictures through TT/DBIx, but one that can be bulk updated when a suitably large change to the underlying data is applied at frequent intervals. The situation where every single item has to have a database hit for no reason is just a waste of (limited) resources.

I've finally got enough familiarity with supporting modules to start applying Catalyst, so the re-write will possibly be my first full foray into it. The ideal would be if this function already exists within Catalyst.

Thanks for looking

Comment on Creating static pages from DB on schedule
Re: Creating static pages from DB on schedule
by moritz (Cardinal) on Nov 24, 2012 at 11:00 UTC

    A common approach for increasing performance is just to use caching. In a way, generating static files is one form of caching, but there are others that you could consider, which might be easier than a rewrite of your application from dynamic pages to static pages:

    In any case you can invalidate the whole cache when there are big updates to your database.

Re: Creating static pages from DB on schedule
by CountZero (Bishop) on Nov 25, 2012 at 08:07 UTC
    If your semi-dynamic pages are made through a single subroutine call (or you can wrap the rendering of these pages in such a call) then Memoize might be something to look into. It has the possibility to "cache" the calls to this subroutine to disk if necessary.

    CountZero

    A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

    My blog: Imperial Deltronics

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1005360]
Approved by moritz
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (7)
As of 2014-11-23 05:09 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My preferred Perl binaries come from:














    Results (128 votes), past polls