Reasons for looking at your favourite module's source

[liz]

Although I agree with Abigail-II's answer to DIY Tie, I truly wonder whether looking at the source code of your favourite module would be the best thing to do to learn programming. Generally speaking, anyway.

Suppose your favourite module is CGI.pm. Ovid already described its coding quirks before. Would you learn something from it? And if you would learn something from it, would it be considered "good" by current coding standards (whatever they are?).

Personally, I always attempt to document my programs / modules in such a way that they may actually teach something to the people actually looking at the source. Why I do this? Habit, really. A habit I thought myself, because I've had the "pleasure" to have to look at my own source code more than 5 years after I wrote it. And the first time I had that pleasure, I was cursing the author (that would be me) for not having documented the source a little better. So I started documenting my source extensively. Some might say, excessively: to them I say, pipe the source through PerlIO::via::UnComment! ;-)

Getting back to the reason for this meditation. I think there are legitimate reasons for not suggesting more or less "naive" users to look at the source of their favourite (standard) module as a learning experience. Here they are in what I think is in descending order of importance.

The Cons

Historical deformation

The code of most of the standard modules was originally written for Perl 5.0, possibly before that. Lack of certain features then caused the code to be written in a certain way (e.g. lack of our) or would execute without issuing warnings (e.g. lack of warnings.pm). New Perl programmers looking at these modules, will take over these bad idioms. Cargo-culting results.

Hardly any internal documentation

Many of Perl's core modules lack almost any internal documentation. And I'm talking the Perl source code components here, not the XS code. That is generally even worse. If you learn something from such a module, you generally only learn "what" it does. Not necessarily "why" or "how" it does it. More cargo-culting results. I see the following reasons why there is not a lot of internal documentation in those modules:

• Execution speed: having to scan through documentation costs relatively more time: that may have been a problem then, it isn't anymore now. In that sense, mod_perl changed the game significantly.
• The initial developers of Perl were even more of the "Those who can, do. Those who can't, teach" types. If you don't understand the code, you have no business seeing it.
• Mistaken productivity. If you don't have to document your code, you can program more lines per day.
• Laziness. Generally considered a good quality when programming in Perl. I consider this to not be a good quality when it comes to documentation, though.

Changed optimizations

For example, in the first versions of Perl 5, method lookup with objects was notoriously slow. Instead of calling subroutines as methods, you would call them as subroutines (adding the object as the first parameter) if you were certain which subroutine would be used to handle the method call. (And yes, I have been guilty of that). Of course, this breaks sub-classing and thus general module re-usability. But it works and was much more efficient then. You don't want people to do that anymore with new Perl code. Still more cargo-culting.

On the other hand, I do think there are a number of reasons why it would be a good idea to expose "naive" Perl programmers to the inside of their favourite modules. In descending order of importance:

The Pros

Exposure to other ways of doing things

By looking at other people's source code in general, you are likely to be exposed to other ways of doing things. Things you knew were possible, things you didn't think were possible at all but turn out to be just a little hard in Perl.

Historical perspective

By looking at older source code in general, you can get an appreciation on how things used to be before they came as good as they are today. And it will hopefully also let you realize that what you consider good now, may be superceded by something much better still in the future.

It's all done by someone, somewhere, sometime

Too many times, users of Open Source software in particular, seem to forget that all the stuff they're working with, has had to be programmed by someone, somewhere in time. And it cost them a lot of work to get it right. Hopefully this will instill some sense of appreciation. We stand on the shoulders of giants, indeed.

Of course, I'm wondering what my fellow brethren would have to say about this matter.

Liz

[adrianh]

Another pro: It acts as a salutary lesson on why you should write good, clear, readable code yourself.

Nothing drives this lesson home more than that first time you have to understand and maintain somebody else's code ;-)

[VSarkiss]

Better still: the first time you find yourself having to fix the code you wrote five years ago, and realizing you have no recollection of why you wrote it the way you did.

[Abigail-II]

The code of most of the standard modules was originally written for Perl 5.0, possibly before that.

That's certainly not true. The uncompressed sources of 5.0 are approx. 5 Mb. 5.8.0 clocks in at over 53 Mb, which is almost a doubling of 5.6.1 (less than 28 Mb). The increase is size between 5.8.0 and 5.8.1-RC4 is about 5 Mb, the size of the 5.0 distribution. Sure, not everything in the distribution is modules, but it does show the explosion of modules in the standard distribution.

Abigail

[liz]

You're right. I guess I'm still living in the 5.00503 era mentally ;-)

Liz

[jdtoronto]

I especially liked the comment about going back to your own code 5 years on. Back in the late 60's I started working in Fortran-IV. This same code is the root of a product which has been through many incarnations in a variety of languages over more than 30 years. Incarnation TWO was a problem, there were no comments in incarnation ONE so I had to go back and work it all out by hand. But since then, each change or addition in whatever language has been thoroughly documented, to the point where the comments are now about 1.5 times the size of the code itself and they provide a wonderful insight into over thirty years of development.

A simple rule:

If you comment well, no amount of comment is too much

And yes, there are Perl coders out here who have been at it that long!

jdtoronto

[BrowserUk]

It's strange, but no matter how many times I see the arguments for "lots of comments", no matter how well argued they are, nor matter whether the proponents are those I have great respect for or not, I always end up with the same thought running through my brain.

If I cannot understand the code without reading the comments, then I need to work out how that code works, for myself, without hints, otherwise I won't really understand it, and therefore risk breaking it when the time comes to modify it. This is true whether then code in question is old code of my own, or code I get from someone else.

For me, this is a little like driving a car without having an appreciation of what goes on under the bonnet (or in the EMU or ABS etc.). Many people are quite happy to be oblivious to the details, and are quite comfortable with just knowing that the feature is there and working on their behalf, but I just have to know how it works.

I realise that this is indicative of (another) character flaw in myself, rather than a condemnation of others, but that's just how I am.

That's why I tend to stick to the occasional comment in my code of the form "This next bit does such and such", rather than copious explanations of every routine or block. These act as flags that draw my attention to those bits of the code that will probably require some effort on my part to understand again in 2 years time. As such, they serve a higher purpose than just explaining the code, they draw attention to those bit that need explanation.

My tendancy to remove overly copious comments from other peoples code and read the code rather than the comments comes from several, painful examples where the comments were innaccurate, out of date, or just plain wrong that caused me personal and/ or profesional grief.

I had one person say that "Of course you shouldn't believe the comments verbatim, you should always check that the code does what the comment say!".

My only response to that is: In that case, what damn purpose do the comments serve?

This is a very personal viewpoint, and not one I expect many people to share, but until someone devises a mechanism for testing the comments along with the code, I will continue to believe only what the code tells me, and will, at best, ignore the comments, but if they are too copious

#######################################################
#                                                     #
#   OOO   RRRR          L       OOO   U   U  DDDD     #
#  O   O  R   R         L      O   O  U   U  D   D    #
#  O   O  R   R         L      O   O  U   U  D   D    #
#  O   O  RRRR          L      O   O  U   U  D   D    #
#  O   O  R   R         L      O   O  U   U  D   D    #
#  O   O  R   R         L      O   O  U   U  D   D    #
#   OOO   R   R         LLLLL   OOO    UUU   DDDD     #
#                                                     #
#######################################################

and therefore distract from my overview of the structure of, and close-up view of the detail of, the important stuff, IE the code, then I will delete them.

My editor macro for doing this in perl code is beginning to get quite clever:)

---

Examine what is said, not who speaks.
"Efficiency is intelligent laziness." -David Dunham
"When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller
If I understand your problem, I can solve it! Of course, the same can be said for you.

[tachyon]

Let the code describe what is being done and comments simply to describe why.....

    # the sort on length means we will delete
    # dir/subdir/subdir/ before dir/subdir/ and dir/
    # which avoids a directory not empty error
    for my $dir ( sort { length $b <=> length $a } @$dirs ) {
        rmdir "$DOC_DIR/$cwd/$dir" or $errors .= "Failed to delete $cw
+d/$dir: $! ";
    }

cheers

tachyon

s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print

[bsb]

perlnewmod:

Step-by-step: Preparing the ground
Before we even start scraping out the code, there are a few things we'll want to do in advance.

Look around
Dig into a bunch of modules to see how they're written. I'd suggest starting with Text::Tabs, since it's in the standard library and is nice and simple, and then looking at something like Time::Zone, File::Copy and then some of the "Mail::*" modules if you're planning on writing object oriented code.

These should give you an overall feel for how modules are laid out and written.

My personal recommandations would be Template::Toolkit and CGI::Kwiki. Two architectures that have impressed me recently.

Brad

[demerphq]

I'm more in the "have a look at the modules source" camp than not. I have learned a great deal from reviewing the code of much of the Perl 5.6 distribution. However I beleive that new users are just as likely to be overwhelmed by reviewing an intense modules code. CGI.pm for instance is not a particularly good place to point newbies to, nor are a number of others (P::RD, or frankly almost anything that Ive looked at from TheDamian :-). However intermediate and advanced programmers can learn a lot from both.

BrowserUk also has a point. I tend to like to have a fairly good understanding how the modules I use work. I dont need to grok it to the level he says he does, but when time permits I often reimplement things that I dont understand as a learning exercise.

I think the question here comes down to different issues. Is the person you are advising a total programming newbie? Are they a decent programmer in something else but fresh to Perl? Are the a student of programming, or a sysadmin looking for a quick hack? Etc etc.

For the record i've recommended a few times that people have a look at Data::Dumper's code, but those were mostly in the context of a specific question.


---
demerphq

_{<Elian> And I do take a kind of perverse pleasure in having an OO assembly language...}

[tilly]

Another pro is that the core code can always use more review. There are bugs, and sometimes they are surprisingly findable.

For instance Ovid also wrote use CGI or die;, which includes among the benefits of CGI.pm that hand-rolled solutions don't tend to test $ENV{CONTENT-LENGTH} to catch aborted downloads. Well it was by reading the source-code that I found that a hard-to-debug problem happened because CGI.pm didn't either on a regular POST. (Only on a multi-part post.)

And even if you don't find any bugs in reviewing a bunch of that code, you will get insight into ways that very smart people who you have never met think. This is never a bad thing (even if the state of the art has progressed).

For a possible, if anyone wants to read Code Reading: The Open Source Perspective and get back on how it is, I would be interesting. I would have read it, but I have been busy learning stuff (eg Oracle) for a new job...

[Anonymous Monk]

Suppose your favourite module is CGI.pm. Ovid already described its coding quirks before. Would you learn something from it?

Would you learn nothing from it? You learn something from absolutely everything. Spend less time lazily debating if it's worth it, and spend more time actually doing it. I have yet to meet a successful person who sits around going "hmm, should I learn this, I wonder how it compares to this..." Stop looking for reasons to avoid challenges.

And if you would learn something from it, would it be considered "good" by current coding standards

It doesn't matter whether the code is "good" or not. As long as you keep an open, objective mind you'll learn just as much, if not more, from it. If you can't handle "experts" telling you flat out lies, if you can't learn to prove things for yourself, find a different hobby.

[demerphq]

Stop looking for reasons to avoid challenges

And where did Liz ever say she wanted to avoid challenges? I think Liz was speaking as an experienced Perl programmer about the question as to whether it is good advice to tell people to examine their favorite module.

You have some points I agree with, looking at CGI will teach you things irregardless, as does any form of experience. You also touch on the question of "good" programming practises. Experts of course are always at liberty to toss such practices to the wind, however teachers for good reason are not, for equally good reasons.

Anway, while you seem to have some interesting points your tone is extremely negative and disrespectful, and frankly smells to me like a personal attack, (after all you snidely imply Liz is not a successful person, which is both rude, and to my knowledge untrue.) So because of this your node has earned my first -- in months. Perhaps next time you will unmask yourself and participate in the forum in a more civilised and gentlmanly way and then we can take your contribution a bit more seriously.


---
demerphq

_{<Elian> And I do take a kind of perverse pleasure in having an OO assembly language...}