Re: Making perlmonks seach engine friendly

[dave_aiello]

I would like to make a couple of general comments about Web application design and search engine friendlyness that seem relevant to PerlMonks.

Last year, I provided a bit of assistance to Automatic Media, the folks that built Plastic.com. IMHO, Plastic is one of the best commercially-oriented implementations of Slashcode ever done, and a real credit to the people who built it, including Joey Anuff and Jon Phelps.

In the course of our discussion, we all agreed that one of the biggest problems with Slashdot was the site's "lack of memory". In early versions of Slashcode, the lack of memory stemmed from the fact that the only way search engines could find content that had scrolled off the home page was through the "Older Stuff" Slashbox, which generally only displayed the links to the story pages for the previous 7 days. If Rob Malda and friends had, at the time of Slash 0.9, chosen to display an paging device (such as link labeled "Last 20 stories ->" at the bottom of the home page), they would have created a searchable site since all story pages were static rendered.

With the advent of Slash 1.0 and later versions, many advances had been made in the efficiency of Apache, mod_perl, mySQL, and in the efficient deployment of these resources across multiple machines. So, the developers of Slashcode and code bases that were influenced by it (like the Everything code bases upon which PerlMonks was built) focused on dynamically rendering as many content pages as possible. This was thought to improve the user experience, which I guess it did marginally for those people who already knew about the Web Sites in question. But, it actually made the site less useable for people who would have otherwise found the site by doing searches on Google and other search engines.

Ultimately, this has hurt sites like PerlMonks more than Slashdot, because the nodes in PerlMonks are of greater lasting value to Perl users than the discussion of current events that is the core of Slashdot content.

If you agree with me that greater search engine accessibility would improve PerlMonks, then one obvious way forward would be to enhance the underlying code base to have it perform more static rendering of nodes. This would not have to be an all-or-nothing approach-- the techniques of server side includes could still be used to merge static and non-static page components when logged in users visit each node. There are definitely other techniques that could be employed, such as URL rewriting that would hide URI parameters in permanent-looking URLs. But, I think increasing the amount of content that is static-rendered would be the most efficient approach, if you look at the site holistically.

Dave Aiello
Chatham Township Data Corporation