PMSI - Perl Monks Snippets Index

[grinder]

Ever run into the situation where you (vaguely) remember seeing a snippet posted about something, months ago, and suddenly you need it urgently?

Such a thing happened to me the other day, and so I started to trudge through the depths of the Snippets Section. O! fateful day! It turns out that there is no easy way to navigate through that section. You add a snippet, it gets added to the first page. When the first page fills up with new snippets, the older ones get relegated to subsequent pages. When I embarked on this task, the oldest snippets were some 28 pages from the front...

Worse still, if on the thirteenth page you have to post a comment to the CB... you go back to the beginning again.

This drove me up the wall, so I decided to do something about it. I set about writing the...

Perl Monks Snippets Index

This provides you with a number of ways of viewing the Snippets section. As the resulting files are all static HTML, it should prove to be search-engine friendly, to boot.

Your comments and criticisms are most welcome. Please do not laugh at my HTML. This is about the best I could do.

Anyway, as they say, have fun.

--

g r i n d e r

[jcwren]

Sweeeet!

And I really like it because it demonstrates an excellent use of the perlmonk.org web-hosing services.

Snippets has needed something like this for a long time. Great job!

--Chris

e-mail jcwren

[stefan k]

Howdy,
first of all -of course- thanks and respect for your work!

Then to the suggestions:

The index by word should maybe be ordered alphabetically within the scope of the initial letter.

The following page when a word is often used could mention the date, if possible (so that in case you can remember the vague time you read that snippet you need now so urgently you get a better chance).

The best contributor page may also take the reputation of a snippet into account. Of course we all know that neither the rep nor the number of postings is directly correlated to the quality, but nevertheless...

And one last remark for the sorted-by-date page: If the perlmonks site will live for a longer time it will probably (and hopefully) become necessary to split the page (maybe into one document per year)

Anyway, thanks for YAUF ^* :)

Regards...	Stefan
you begin bashing the string with a +42 regexp of confusion

^* Yet Another Useful Feature

[seanbo]

Very nice! Something that might be cool on the Author sorted index...Links to display those that contributed 4 or 3, etc rather than just saying how many there were. Just a thought. Either way, very cool. ++ to ya grinder!

seanbo

[wufnik]

quick note - i am as big as fan as any of your snippets index, for which you should be sainted, but i don't visit every day - could an argument be made for a 'snippet a day' mailing list, along the same lines as the perl FAQ a day?

a snippet a day keeps your problems at bay etc etc etc...

wufnik

in the world of the mules there are no rules

[mischief]

nice one! very useful.

any chance of adding a search feature? how does it work - can we see the code?

[grinder]

In my browser, I get a search feature by holding down <ctrl> and pressing F at the same time :)

How does it work? It is a series of scripts that have to be called in the right order. It's a bit messy, because it was very much an exploratory process.

The first pass is a script that just keeps walking down the chain of snippet pages until it can't find any more links. When it is run the second time around, it walks down the pages until it encounters a link that it has already seen. It sleeps 15 seconds between fetching each page (Perl Monk HTML hackers would do well to take notice of that last point). Similarly, the process is cronned at 4:15 UTC as I figure that's a pretty quiet time for yoda (the machine perlmonks.org is running on).

A second script then kicks in which cleans up some yucky inconsistencies, like reaped nodes with no titles, and reformatting the data to make it easy to process afterwards. This was a bugger to get right. I first tried to do it all in the first script, but it turned out to be simpler to let the fetching script do as little as possible, just fetch and dump, and let another script do the cleaning. It's awkward to carry state around between HTML::Parser callbacks.

A third script then takes the cleansed file and loads it into different hashes, to print them out in various sorted orders all different, for the various HTML views. For instance, it's at this stage where I calculate the number of nodes a person has written, how many nodes written by monks whose nicks start with 't', which makes it easy to emit the correct rowspan attributes to get everything to line up.

A fourth script then walks through all the files generated by the third pass and encodes them as HTML. No, I didn't use any HTML-generating modules. Naughty me, I did it all by hand, serves me right for not paying attention to what modules jcwren has installed on the server. This script creates the files in a directory named /pmsinew under my document root, and then when it has finished, it names /pmsi to /pmsiold and /pmsinew to /pmsi, and then proceeds to unlink the /pmsiold directory. Which means it should be pretty hard to come across a half-constructed index.

It would be a euphemism to say that the code lacks elegance in certain places. I was more interested in hacking up something quickly than showing The Right Way to do things. I do promise to rewrite the scripts and maybe even drop in a comment here and there.

When I do, I'll post the link from the /pmsi/ homepage.

--

g r i n d e r

[muba]

Nice job!
It even lists a snippet posted by me, which I didn't even remember!

[mikeock]

As of today the link above is returning a 404 message.