Re: Are monks hibernating?

[jdporter]

Well, I was doubtful of this. It's very easy to see trends when there really aren't any. So I decided to do some research.

I looked at the rate of posting of root SoPW nodes since the beginning of the (PM) epoch. Here is what I found.

First, the rate of posting has been very consistent throughout almost the entire history of PerlMonks. This in itself is interesting, because it could mean that interest in Perl has been flat, in the aggregate, despite developments in the product and the advancing competition. (Perhaps those factors balance out.)

Second, there are at least two cycles visible in the rate of activity, one weekly and the other annual. This should come as no surprise.

Third, beginning in March of last year, there has been a steady decline in the rate of SoPW postings. Just eyeballing the graph, trying to smooth out the various cycles, it looks like the current rate of SoPW postings is about one half of what it has been historically. I didn't do any analysis of the last 2 to 3 months specifically, but the trend has been fairly smooth, and we're now at the lowest level of SoPW activity since about January of 2001.

I don't pretend to know what any of this signifies.

[tye]

Thanks to you (and others) for looking up some numbers.

I'm pretty convinced that the recent decline is due to me finally closing the back door where search engine spiders were (badly) indexing the site. Note, however, that bad indexing has at least some advantages over no indexing. The back door was closed for good reasons and it was thought that a long-running project to produce search-engine-friendly renditions of pages would be finished "RSN". Also, closing the back door doesn't appear to have magically ended our recurring problem with one of the web servers going "out to lunch".

So, in the short term, we should probably re-open the front door (but keep the back door closed) and try to only open one front door (just www.perlmonks.org, not perlmonks.org nor www.perlmonks.net etc) so that we have "okay" indexing. It will be unfortunate that snippets of CB content will be indexed and cached and some of the features for making it easier to do more powerful searches via google (et. al.) won't be there (my plan was to add keywords to pages so you could tell google that you only want to search a specific section or only for a certain author even if that author's name is something heavily used like "grep"), but at least we'd be on the map and "strangers" might find some of our useful content.

I'll put that near the top of my to-do list.

[tye]

Okay, say "hello" to our new (dynamic) /robots.txt; what you see there will depend on what hostname you use to visit PerlMonks. Compare http://perlmonks.net/robots.txt vs. http://www.perlmonks.org/robots.txt.

Now we see how long it takes google, et al, to notice and then wait for the traffic levels to rise until the site becomes just annoyingly slow enough that we reach equilibrium (which I think explains the quite flat site traffic level prior to shutting out the search engine spiders).