http://www.perlmonks.org?node_id=158520


in reply to Improving performance by generating a static html file?

I have built, or participated on the build of at least two sites that use a generated static HTML approach, with good results. None of them in perl, but that should not matter for this discussion. :)

One of the sites use the approach that it will regenerate the appropriate pages when the administrator updates something - this can involve adding a new news item, or adding a new object for customers to view. This has pretty low frequency, and for all practical purposes, there is only one administrator, so there are no concurrency issues either (although safety has been built in in case they would suprisingly expand a lot or something). All the changes made are saved into a database, and upon pressing "publish", recently changed pages will be regenerated. Since visits are really plentiful when compared to the updates, this is an approach that works really well. Matter of factly, this kind of approach could mean you could have a site on a host with no dynamics at all - you generate the pages from somewhere else and put them there via ftp, scp, rsync or whatever is appropriate. :) In this case though, there are dynamic parts, which includes a browsable web map and customer logins. These parts are purely dynamic, partly for session stuff, and partly because this is still low traffic parts.

The other approach is an hourly update from the article data base, to the web shop. Not the same company though. When a user buys something, or searches etc, dynamic pages are generated directly from the database, but when they are just surfing around, they see static content. This has some issues when dealing with discrepancies between static and dynamic content upon rare occassions, and we are not satisfied with the solution. However, we picked a bad platform for the job, it couldn't handle the huge load any other way, and we hopefully know better now. Mistakes are good for learning, if not for confidence... *grin*. And no, I am not allowed to fix it. :(

Anyhow, given the right circumstances, those two approaches really works well when you are worried about heavy load.

Your third suggestion was to query the DB, or disk, every time someone wanted a page and either return a cached page, or generate and return a new one, right? Well, I still advocate updating on change instead of polling for changes, given that changes are more rare than views. Then the views are without any load at all, while the fewer updates have some load (but not a poll for changes).

However, depending on the expected load, I am not sure that any static HTML approach is fitting for a forum. If load is high enough, you will have lots of concurrency problems, and you will need to lock the files on disk and other such things - effectively slowing things down a lot again, while they wait for each other. If load is low enough for this to not be a problem, then load is probably not a problem for generating everything on the fly either.

It would still depend upon the forum, or whatever, but the smaller the difference between updates/reads, the worse is a static HTML approach imo, no matter what the expected load is. And forums are generally something that gets updated pretty much compared to just viewed - unlike a news site or similar. :)

If you have the time and possibility - try both approaches (static and dynamic). Otherwise, try to think what you can gain/risk/lose on either approach or a healthy combination.

This probably won't help much. I still hope it will. :)


You have moved into a dark place.
It is pitch black. You are likely to be eaten by a grue.
  • Comment on Re: Improving performance by generating a static html file?