Re^4: Perlmonks site has become far too slow

[soonix]

in the CB you mentioned "ticket system" and "SQLite" very close to each other recently, which reminded me of Fossil's ticket system, using (and used by) SQLite.

Their Defense Against Robots works thus:

• (of course, complemented by other measures)
• most dynamic links are shown to logged-in users only
• there is an "anonymous" login (with dynamic password) for anonymous humans

Of course the design principles of Everything₂ and Fossil are different enough, but I like the concept of an "anonymous login" 😉
and most (perhaps all?) of the reasons why PM still allows anonymous posts would still be respected by this…

[jdporter]

That sounds like it could be viable. But, help me understand: what is meant by "dynamic links"?

One thing to remember: we don't want to completely prevent robots from spidering the site; we want people to be able to use g0ggle (etc.) to find information in perlmonks. We're just looking at ways to throttle robots so they don't (inadvertently or otherwise) DDOS the site.

[soonix]

From the text there, it sounds like more or less any link. However, when looking at the site without logging in, there are a great number of valid links.

For Perlmonks, it would probably be the Super Search, RAT (at least over a defined nesting depth), and nodelets with dynamic content (e.g. Other Users, CB) that would be "unlinked". But I think these are already blocked for robots.

So, probably not much to gain here :-/ especially given the fact that this concerns only "A HREF"s, not incoming links from elsewhere (or self-generated ones).

we don't want to completely prevent robots from spidering the site

Yes, that's understandable. Don't know how Fossil handles that. OTOH, they seem to be less well known, maybe as a result from exactly that 🙈