Looking for examples of well-written modules

[xdg]

Dear fellow perlmonks,

I'm looking for a list of well-written Perl modules.

I think of Perl as a very "literary" language. Between Larry Wall's focus on syntactic convenience and fluidity, the principles of TIMTOWTDI, and the plethora of idioms, Perl is written in a wide range of styles unique to individual authors. There is "Baby Perl" of course -- like the "See Spot Run" type books I had as a child. And it wouldn't be hard (and could be an interesting meditation) to make further literary analogies as well: code like Hemingway written in short choppy functions or code like Austen written in florid, overly commented and meandering code. (No offense to Austen fans intended.)

Just as I learned to write in English at least in part by doing a lot of reading, I feel my Perl writing would improve by more exposure to other people's code, particularly, to well-written code. I'm looking for pointers to particularly well written modules (or open-source perl projects) that demonstrate good technique across a range of styles.

In scouring PM and other places, I've not found much that addresses this directly. I'm not looking for a list of "great" modules. Typical answers to this kind of question can be seen on Learning from modules -- which suggests looking at favorite or popular or otherwise "great" modules. I love lists of cool modules like the Advent calendar, too. But cool or heavily used or popular modules are not necessarily well-written or good to learn from -- even though they may be quite functional for day-to-day usage. They may have some quick and dirty hacks inside, may be poorly documented, may be full of "cargo-cult" holdovers, or may be so idiomatic as to be nigh-impenetrable.

Just as I wouldn't point someone to Barnes and Noble and say, "go browse around some bestsellers" to someone looking for a well written book, being told to go look at popular modules on CPAN isn't particularly helpful and is something I could have figured out myself.

Perl code in books about Perl isn't bad but isn't too great either, as it tends to be written simply for a wide audience or tends to be idealized to demonstrate particular techniques or situations. (An exception is probably "Effective Perl Programming" which I'd analogize to Strunk & White "Elements of Style" for improving one's craft).

All of which is to say that I'm looking for thoughtful suggestions for well-written real-world code that is available for review. This could include (but shouldn't be limited to):

• Other people's code (modules/projects) that you've crawled around the inside of and think is well done
• Good articles/presentations/books with fuller descriptions of working code
• Things you've written yourself that you'd self-nominate for closer public scrutiny
• More broadly, any CPAN authors that you feel generally demonstrate good style and technique

(To go one step further I'd also be interested in hearing the same about test-suites in modules. Now that I'm trying for a more TDD style, I'm pondering if I'm writing too many or too few tests in certain situations and good examples would be helpful.)

Thanks very much,

Edits: Fixed spellings for Austen and Hemingway -- thanks Cody Pendant. My degree was clearly not in English Lit. :-)

[webfiend]

The problem with reading code as literature is that you want to be able to guess the ending of good code. ;-)

"What? return 1;? I didn't see that one coming. I thought $butler->did($it);"

Oh, and there's also the problem that I am usually looking at code to find out why it's not behaving the way I want. Not exactly conducive to a pleasant reading experience. Still, any Damien Conway module makes for a good read - sometimes in a William S. Burroughs "Naked Lunch" sort of way, but a good read nevertheless.

[Cody Pendant]

I wouldn't normally do this, but I really have to point out that you mis-spelled the names of both Ernest Hemingway and Jane Austen.

If indeed that's who you meant, because Austen wasn't known for her writing being florid or meandering, quite the opposite. For her time she was practically telegraphic.

Here's the opening to Pride and Prejudice:

It is a truth universally acknowledged, that a single man in possession of a good fortune, must be in want of a wife.

And here's a sentence from right at the start of Sense and Sensibility:

The old gentleman died: his will was read, and like almost every other will, gave as much disappointment as pleasure.

A much better choice of meandering novelist would be Henry James, who descends so often into sub-clauses and sub-sub-clauses that you long in vain for some curly brackets to break the thing up into nice blocks:

She had got up with these last words; she stood there before him with that particular suggestion in her aspect to which even the long habit of their life together had not closed his sense, kept sharp, year after year, by the collation of types and signs, the comparison of fine object with fine object, of one degree of finish, of one form of the exquisite with another--the appearance of some slight, slim draped "antique" of Vatican or Capitoline halls, late and refined, rare as a note and immortal as a link, set in motion by the miraculous infusion of a modern impulse and yet, for all the sudden freedom of folds and footsteps forsaken after centuries by their pedestal, keeping still the quality, the perfect felicity, of the statue; the blurred, absent eyes, the smoothed, elegant, nameless head, the impersonal flit of a creature lost in an alien age and passing as an image in worn relief round and round a precious vase.

Now that's meandering...

Guess what my degree was in?

[bradcathey]

Cody Pendant, how can you forget George Eliot and her comment about marriage (Middlemarch—considered by many as the greatest novel in the English language),

"Marriage, which has been the bourne of so many narratives, is still a great beginning, as it was to Adam and Eve, who kept their honeymoon in Eden, but had their first little one among the thorns and thistles of the wilderness. It is still the beginning of the home epic—the gradual conquest or irremediable loss of that complete union which makes the advancing years a climax, and age the harvest of sweet memories in common."

(Even music majors can read ;^)

As to the OP, what makes great code?? Style, speed, brevity, cleverness, *or...*usefulness, practicality, applicability? George Eliot, Henry James, and others mentioned here, were some of the greatest technicians to ever take up the pen, but they also had tremendous insight into human nature, religion, the social condition, history, etc., and were able to communicate it persuasively.

I love great code as much as the next monk (some of the answers I've gotten here at the Monastery have made me just smile and shake my head in amazement), but my clients never see it my code. They just want an intuitive interface that makes sense, gets the job done, and doesn't break the bank. Yes, well-written code helps to that end, but as in most disciplines there is always a balance of craft and artistry, of genius and pathos.

We all strive to write the best code, but we also need to serve our users. Let's not let one get too far ahead of the other. I can never get it right, is it form follows function or function....

Update: After I wrote this I got to thinking that there are times in both my design work and my coding that I do things for the sake of doing it right, even though the customer will never notice—like taking the time to kern the letters of a headline or tighten up the scope of some variables in a script. I guess we do some things simply because it's the right thing. I've heard it said that character is what you do when nobody's looking.

[rdm]

It has to be said.

If you want a truely meandering style no-one beats Edward Bulwer-Lytton, with his now infamous quote from Paul Clifford:
"It was a dark and stormy night; the rain fell in torrents--except at occasional intervals, when it was checked by a violent gust of wind which swept up the streets (for it is in London that our scene lies), rattling along the housetops, and fiercely agitating the scanty flame of the lamps that struggled against the darkness."


Now, as to the quality of that work...that is another matter.

[Cody Pendant]

I think my quote from James still stands head and shoulders above those two examples.

We're talking 932 characters, 165 words. George Eliot was positively monosyllabic in comparison.

This is actually on-topic, because I used Lingua::EN::Sentence to parse the whole Gutenberg Edition of James' "The Golden Bowl" to find the longest sentence.

I would happily re-write it in mock-Perl syntax, but after many readings, I still don't actually know where the clauses begin and end, let along what it actually means...

[pernod]

Ah, but can we talk of modern literature without mentioning Marcel Proust? From the opening paragraph of Du côté de chez Swann.

Je me demandais quelle heure il pouvait être; j'entendais le sifflement des trains qui, plus ou moins éloigné, comme le chant d'un oiseau dans une forêt, relevant les distances, me décrivait l'étendue de la campagne déserte où le voyageur se hâte vers la station prochaine; et le petit chemin qu'il suit va être gravé dans son souvenir par l'excitation qu'il doit à des lieux nouveaux, à des actes inaccoutumés, à la causerie récente et aux adieux sous la lampe étrangère qui le suivent encore dans le silence de la nuit, à la douceur prochaine du retour.

The same passage in english from the english translation on gutenberg.

I would ask myself what o'clock it could be; I could hear the whistling of trains, which, now nearer and now farther off, punctuating the distance like the note of a bird in a forest, shewed me in perspective the deserted countryside through which a traveller would be hurrying towards the nearest station: the path that he followed being fixed for ever in his memory by the general excitement due to being in a strange place, to doing unusual things, to the last words of conversation, to farewells exchanged beneath an unfamiliar lamp which echoed still in his ears amid the silence of the night; and to the delightful prospect of being once again at home.

Now that's what i call meandering :)

[giulienk]

I was about to bring up Marcel Proust myself, but you already did and with a good bilanguage quote, good job :)

[Anonymous Monk]

If you are looking for long sentences check Durrenmatt. He wrote a novel in 9 sentences. Each chapter is a sentence. Looks like a modules without functions. So maybe all discussion comes to how many functions (sentences) you have in a module (chapter), and how many interesting ideas (idioms) you have in a sentence. Objects maybe related to plots or characters (actors). Just my 2 cents

[qq]

The English Novel - pre 1912

A fool and his fortune are soon married.

    -- J. Chassler

[merlyn]

• Good articles/presentations/books with fuller descriptions of working code

Well, there's always my 200+ magazine articles.

Oh, you said "good"? {grin}

And if you want the best of them in "dead tree form", you can get my new book from Apress, Perls of Wisdom.

[tilly]

I notice that nobody seems to have actually given you real modules to read.

Let me rectify that.

I personally learned a lot from reading the source to LWP. While CGI is an important module, I can't recommend it as good example of how you to code. Modules written by TheDamian impress, but in a fashion that you're likely to find impossible to emulate. Still read a couple anyways.

Beyond that, if anyone has impressed you (like Simon Cozens), grab a few of their modules just to read it. Afterwards you may wind up more or less impressed, but you'll learn something.

But I still think that the cleanest coding style that I have seen had nothing to do with Perl. It was when I peaked at the source to Ruby by Matz. The addition of maintainance since then (and not all by Matz) might have muddied it.

[stvn]

I'm looking for a list of well-written Perl modules.

This is a very difficult request, because of TIMTOWTDI one mans well written code is another mans crap. Personally Austen puts me to sleep and I think Hemingway can be a pompous homophobic ass, sure they can write well, but the subject matter and style don't appeal to me. I would rather a whirlwind of sex and insight with Henry Miller or a good drug-induced paranoid romp with Phillip K. Dick or enjoy a good graphic novel by Joe Sacco or Art Speigelman or one of my all time favorites Alfred Bester whose work always leaves me wishing he had written more. But hey, thats whay I enjoy reading, which is just a highly subjective opinion coming from a complete stranger.

Code I think (as you indicate as well) is much the same. I personally enjoy reading code, I get a lot of satisfaction from it and I agree with you that I find it a valuable learning experience. But given the choice I like to read code in functional languages (or code written in a functional style). Standard ML is a favorite of mine, along with Erlang and sometimes Haskell (LISP and Scheme just have to many parens to make them enjoyable to read). My code, although almost always OO, leans towards that functional style too. To those of the more imperitive persuasion, this may make my code akward and difficult to follow. Some people out there like reading C, personally it just gives me a headache (although I find I have to do it a lot, so I am getting more used to it).

So I guess my point is that you may have a hard time finding good modules to read if you go looking for them. I would instead suggest that you try reading the modules you use often first. Since you know what they do from the outside, you are better prepared to understand them from the inside. And don't worry so much about the good code/bad code thing as sometimes reading bad code can be a valuable learning experience too. And just like books, if you find an author you like, you can try reading their other works too.

... Things you've written yourself that you'd self-nominate for closer public scrutiny.

I was a fine arts painting major and we would many times have daily critiques and I used to really enjoy when I got blunty honest yet constructive criticism. So I don't mind throwing my work up for public scruitiny at all, here is my page on CPAN. I think I am the most happy with Tree::Binary right now, but that might a honeymoon thing since its my most recent one. But anyway, feel free to fire away.

[hv]

This is a very difficult request, because of TIMTOWTDI one mans well written code is another mans crap. Personally ...

I agree, and I think that regularly reading and thinking about the code/text of a wide variety of authors will improve your command of any language.

An interesting approach might be to take a few example modules with which you are not familiar, throw away the author's documentation (the pod that is, not the comments), and then try to write the documentation for the module based on the code and any comments.

Doing this you could hope to learn a lot both about good and bad code style and about good and bad documentation style; maybe also a bit about what aspects of a module should or shouldn't be documented. If you're doing it right I'd expect you also to spot some bugs in the process, which will give you a good opportunity to pay some tuition fees in the form of bug reports and patches. :)

Hugo

[stvn]

An interesting approach might be to take a few example modules with which you are not familiar, throw away the author's documentation (the pod that is, not the comments), and then try to write the documentation for the module based on the code and any comments.

Interesting idea, a little on the insane side, but nice idea. You might even want to just find a module with little or bad documentation, and email the author to see if you can re-write it for them.

Another good way to get to know a module is to work on it's test suite. The Phalanax project is a great place to do this. I spent a week or so a little while ago working on modernizing the DBI test suite, and in the process learned a lot about how that module is architected.

[danderson]

I can't resist. Joyce's Ulysses, considered a masterpiece, with stream of consciousness containging neither punctuation nor comments, might imply that this section's obfuscated code section is worth reading. Take what you will from e. e. cumming's lack of capitalization.

[DrHyde]

Joyce's Ulysses, considered a masterpiece

You missed out the word "incorrectly".

[Your Mother]

++ Testify.

[johnnywang]

I couldn't help adding some authors of non-English master pieces. Well, may be not.

Just want to add/second some thoughts:

• I aslo found the similarity between coding and other forms of writing. When someone says "I can only write, not read", we know it's a joke. But I've seen so many, actually maybe a majority of software engineers, who never ever read someone else's code. I always wondered how they can grow. The greatest thing of the internet and open source for software engineering is that it provides a free school that is better than that ever existed. Yet many of us fail to take advantage of it. So I appreciate the desire to read good code.
• Like reading, I believe the capability to appreciate a good style grows with time. For coding, one just needs to make enough mistakes, and even write enough bad code to be able to write good ones. It helps a lot if there is someone looking over your shoulders.
• PM is probably one of the best places to get a glimps of what some good codes look like. Like books, most examples are short, but one can get the essense.

[Your Mother]

PM is probably one of the best places to get a glimps of what some good codes look like. Like books, most examples are short, but one can get the essense.

This is doubly true b/c all the code that gets posted is defacto peer reviewed. So if something is awkward, overly inefficient, or just plain wrong, it tends to get exposed as such immediately. Sometimes when something is posted that looks great at first blush but is new to me I wait to see what one of the big cats like tilly, dragonchild, merlyn, et cetera have to say about it before I put it in the Bat-belt.

I also think short examples encourage modular thinking and coding. I probably learn more and pick up better style from reading other hackers' subs than their full applications.

I picked up quite a bit from reading the Class::DBI modules. Like using Class::Accessor and some DBI tricks I didn't know. IIRC TheDamian recommends reading anything from Gurusamy Sarathy.

You could also just tip-toe through http://cpanratings.perl.org/ and read the code for anything with 4.5+ cumulative stars. Code that users like is worth checking out even when the man behind the curtain doesn't always measure up face to face.

[jacques]

I'm looking for a list of well-written Perl modules.

http://search.cpan.org/~accardo/

Duh.

[Anonymous Monk]

Funny

[TomDLux]

No, we said readable.

[danderson]

Oooh, I really can't resist.

If we're allowing other languages, let's consider Camus. The code that would parallel his writing would care not what it does, or how it die()s, but only that it exists, and that it is true to itself.

And, well, what of War and Peace? 10,000 line modules must be classics!

[chanio]

What caught my perl newbie attention most is this script: Start.pl. As well as others that are offered for free by some sort of perl programmers in the WWW as a promotion of their names.

They use to do it all without counting on any additional PM. They look like the first people that came to live to America.

In this script case, she encloses all the code inside a HERE that is assigned to a variable.

Finally,

undef($@);
eval $all_code;
if ($@) {

and throws any error to the screen...

This is a tough way of programming, but I guess that everybody can learn a lot of it! Please, have a look also at the full documented page. That is also an important part of a good script, I guess : to be well understood.

This won a price some time ago, for being simple and very useful.

(Extreme programming?, nah!)