The Module Mantra

[tachyon]

Perl Modules save time and effort. They often provide reliable well tested code written by an expert in that area. Better still they are freely available. CGI, LWP, DBI, HTML::useful_widget j(ust to name a few) are all invaluable. But how do you learn Perl, parsing and protocols if you always rely on modules to do the dirty work?

I think that sometimes we are perhaps a little too quick to point to a modular solution. Trying to reinvent the wheel is (in part) how we all come to learn. After all the modules we use would not exist if no one took on the challenge of building a better moustrap. Success is not always achieving the desired end result with the minimum effort - it just seems that way at work :-) Sometimes success is in the learning process.

If we always leave everything to the experts (and their code embodied in the form of modules) where will the next generation of gurus come from? Are we in danger of encouraging Perl coders to be no more than cut and paste script kiddies?

cheers

tachyon

s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print

[bikeNomad]

I think that our quickness to point out module solutions has several explanations:
• Often, someone is asking "how can I do something". We know that the most expedient way is to use something that's already written.
• Even if the question is really "how is something done" instead, modules can be good to point to. I don't know about you, but I've learned a lot from reading the code of the Perl modules. I don't generally use a module until after I've looked through it; why miss a chance to learn? Besides, I've found un/under-documented (or broken) bits of various modules in this way.
• There is a place in the Perl community for the cut'n'pasters. It may be surprising for you or me, but some people don't really want to be programmers; they're just focused on getting their jobs done. For these people, a simple cookbook solution is a good place to start.
• Even if you're using modules, you still have to know some Perl to glue them together.
• The next generation of gurus will not be the ones who are cutting'n'pasting. They will be (as they are now) the ones who dig into the solutions and the possibilities of the langage and the great work that others have done.

[ichimunki]

This is a chicken and egg question-- and there is no answer to it. There is no worry that we won't have a next generation of gurus. We even have a Monk who is quickly going from teenage whizkid into adult guru-- just plain using the regex engine in Perl wasn't good enough for japhy, so he started hacking perl source files.

At some point, any serious user of a module or language feature is going to question the module's or feature's behavior or want to do something a little different. Through the miracle of open source, they can look at what they've been using, see how it worked, perhaps improve it, perhaps decide that it looks too kludgy for comfort and gut it (see Perl 6).

But on the other hand, there are those of us who want to keep our scripting tasks manageable, and a quality module is often a key aspect of doing that. Understanding of CGI transactions is important if you are going to use CGI.pm for anything serious. But spending a lot of time writing all kinds of CGI transaction handlers from scratch is a waste of time, unless you are already well versed in CGI.pm and find that it has a fundamental design flaw that no one ever noticed before. Sure, it may be good as an introduction to CGI to have a quick session of writing simple CGIs without CGI.pm, but once you've done a few basic "hello worlds", why continue down that path, unless your intent is to be a future CGI.pm hacker?

The question of when to recommend a module isn't, IMHO, centered on the learning process, it is centered on the quality, ubiquity, and coreness of the module. Not using CGI is silly, that's like suggesting that instead of using a hash, that you build your own associative arrays out of lists (or something). Why rewrite the stuff that comes with a standard Perl install?

But a module like Tk::HTML is a different story. Is that a module you would recommend? It has no documentation, the sample code included with the module failed to compile on at least one system of mine, and it crashes on tags it doesn't understand. I wouldn't recommend it to anyone who wasn't willing to take the time to rip it apart, to understand Tk and presentation of HTML. For someone who wanted to build a Tk web-browser I might recommend LWP and HTML::Parser (or ::TokeParser) instead, since the amount of work to get to a place of understanding and functionality might be about the same. But certainly they should know about the module so they have it as a reference.

If someone doesn't want to use a core module, though, they should know why. The existence of the core module isn't going to prevent the curious from investigating or trying their own hand at something. I like to see recommendations for good modules where one exists. Obviously if all we've done is search CPAN for something that looks like it might fit, we have not done enough wrt to that question and answer.

[E-Bitch]

I actually have come across this before. I am currently a senior computer science student at a university. Not trying to sling any mud, just making an observation, but during the introduction to programming courses, our courses contained computer science students as well as information systems (a business major oriented towards minor programming). The major observation that I made about these I.S. students is that they could not grasp how things worked on a conceptual level. I would be willing to bet that it was because of the same reason mentioned by tachyon, where C.S students already had a background in programming, and understood (conceptually, at least) what was going on, the I.S. Students were being told how to do things (in java, say, "I need to see how many elements are in this Vector", and they were told, use vecname.size(), instead of being told that the vector (at least in java) has an underlying array, and when you call vecname.size(), you are actually counting how many elements are in that underlying array. (a poor example, but it illustrates the point)).

Anyway, I think that if we want good programmers, we need to teach them what the hell is going on before expecting them to be able to craft an excellent program (that sounds pretty basic, but you would be surprised at how many times I have heard stupid stuff come from people). The people are not at fault, it is the programming ideology. We cant expect people to learn if we dont teach them.

Thanks for the vine!
E-Bitch

[scain]

I suspect most people learn Perl without using modules at first. Certainly my Baby Perl was very C like (even though I was never a very good C programmer), and I always "wrote my own." For instance, I wrote a BLAST paser the the people over at BioPerl wrote better, and I've written several html parsers that mostly worked. For me, it took a while of writing Perl to realize that not only that CPAN existed, but that it could really, really help me. And it was even longer before I realized the Perl Monks existed, and I expect it is that way for the majority of Perl coders.

Granted, we do get a few complete newbies here asking questions that could be and usually are answered with GCC (that's Go Check CPAN). Are we doing them a disservice? I suppose that is a possibility, but all it really means is that they will learn Perl differently from how I did. I never used strict before I came here, so the complete newbie will use CPAN modules but also use strict, so it is a trade off. The fact is, eventually the newbie will have to do something that is not found at CPAN and will have to write something from scratch, and be enlightend.

Scott

[shotgunefx]

Modules make my job easier and my work more robust. I don't think encouraging people to use them will result in stunted programmers. People who like to hack, will. (I routinely look through module source to understand, or try to anyway, what's going on under the hood.)

A bigger concern I have is that when people post that they are trying to create a CGI parser or something that has been done well already, a lot of people jump all over them. I don't agree with this at all. Sometimes you need to re-invent the wheel just for the hell of it.

-Lee

"To be civilized is to deny one's nature."

[tilly]

The most fundamental point that I think that programmers need to learn is how to build on existing work. That is, how to build on their own work, how to build on other people's work, how to write work that others can build on. While there are many skills that programmers need, I think that this one, more than any other, is the lesson that I want people around me to know.

This skill leads to better programs with fewer opportunities for bugs that can be written more quickly. And if you are working on the same kinds of problems for a while (which people often do), it leads to ever increasing productivity for the team.

This is the lesson that I want people to learn from using modules. First of all, using modules written by good people requires learning how to use what other people wrote. If you read the documentation you will often get useful programming tips. Furthermore if you think about the API chosen, and how you might do it differently or why it was done like it was, you have excellent examples to learn from and aspire towards for figuring out the APIs that you create. And when you can try to read the source to those modules, you will learn even more.

Now it is true that simple end users will not do all of that. There are definitely people who can accomplish tasks through cut and paste techniques using CPAN. I don't think that such people get the full benefits of the modules. But at least I can console myself with the fact that they produce better programs with less work than they would without the module.

And for people like me, while I might have learned more about, say, http by writing my own version of CGI, I learned more about programming from reading and thinking about the documentation of modules and Perl than I would have ever figured out from handrolling solutions to a million problems which are readily solved by an existing module.

So I don't see the cry of "CPAN" as just encouraging cut and paste and discouraging the emergence of future gurus. Instead I see it as making people more productive now, and shortening the path for motivated ones to aquire real expertise.

[inverse]

There is enough useful software that is not yet written that no practicing programmer is in danger of running out of real work because he aggressivly uses modules from the CPAN.

We should be pushing newbies to practice making the software they do actually need to write reusable. That way, they can contribute their own modules to CPAN, while becomeing better programmers.

[Anonymous Monk]

It's possible that people do point out modules to others, I think there are a few reasons behind that. One of the main important reasons is the re-invention of the wheel, in which you mentioned. I don't advocate using scripts or other people's code without understanding the behind the scenes. If that means coding something similar to a module or writing one's own routines, so be it. But, overall I do think that if there is a module out there that has proven to be sound to use; I don't think it's wrong to use it. As long as you don't use it blindly. I agree in the sense that one should know the 'workings' before applying a module to their program. tachyon, you have to remember... that everyone who has created a module for use to the world.. has one time been a programmer hacking out their own solution to a problem. One that they found useful and possibly thought others could share from their experiences. Those modules improve over time.. why? Because others use/experiment/play with them in their code and find shortcuts, loopholes, etc. in them.