Re: Loads of Nodes

[FamousLongAgo]

Since we're a programming site, it seems good to solve the problem with a hack: why not have the site run a search automatically, based on the content of the post? That way, it could show a list of potentially relevant nodes in the preview page, before the user submits the question, giving him/her a chance to find the answer immediately.

If you build a term-vector model search engine, running natural language queries like these is easy and relatively fast. I'm working on a HOWTO for doing it in Perl, but I'm sure other monks (especially those who built this site) have at least as much experience.

I know one of the main obstacles to me using the search is that the site is slow - pageloads take a long time, no matter how fast my connection. Does anyone know where the bottlenecks are, and what we could do to improve the system?

Anyhow, requiring users to search before posting seems a little too Nazi-ish for this site; it doesn't fit in with Perl culture. Whenever many new people come to a site like this, it is a struggle to acculturate them to the local values (like - don't post a question without showing at least some token effort towards solving it yourself). But that doesn't mean we don't want them coming to the site - it's just a burden of success. Requiring a (slow) search will surely not help there.

I've been using the site for several months, and I'm only starting to really get the feel of the place. So I try to be forebearing with people who don't know better, especially when they later turn out to be kids, or non-native speakers of English. I still find the number of really good questions (and answers!) makes up for the minor irritation of all the less interesting, write-me-a-program posts.

[Aristotle]

Does anyone know where the bottlenecks are, and what we could do to improve the system?

A lot of the bottlenecks are known, from what I gather - however, there's only so many people with indepth knowledge of the site's innards. Despite the pmdev group's size it seems to be mostly tye, ar0n, vroom (who rarely has any time anymore) and a select few others who have the necessary expertise. And all of them are rather busy running their lives as well. The situation is slowly improving though ("slowly" being the keyword).

Makeshifts last the longest.