Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris

What training do YOU need?

by deprecated (Priest)
on Oct 20, 2002 at 17:08 UTC ( [id://206678] : perlmeditation . print w/replies, xml ) Need Help??

I've noticed recently at $current_client that the developers they have (who develop in perl, but now more and more in java) are completely incapable of new development. Basically, they have been maintaining software for the last few years. Very few of their applications use subs (some of these are thousands of lines long), and none of them use modules.

Recently, they came to the conclusion that perl wasn't really the language for them. The reasoning for this was that their code was awful. It was not portable, it was incredibly difficult to maintain, and it was difficult to keep versions straight. Rather than examine their developers, they chose to point fingers at the language. I think I've told this story before.

Anyways, they are starting to realize that they cannot just develop everything in Java. Something has to process the gigs of web logs and syslogs they get daily. Somebody has to merge databases, and so on. There are all these tasks at which perl excels that Java simply isn't the right tool for.

Slowly they are beginning to realize that their developers are becoming a liability. They cannot continue to bring on consultants (they are paying over $150,000 a month for development consultants right now) to write their software for them. Their only real recourse right now is to fire all their developers (not likely) or to train the ones they have.

So I am attempting to come up with a few outlines. All of the training I've done has been informally. I haven't come up with an outline for a course and in-class materials. My hope is to come up with a general outline that will work for this client, so that I will be able to offer it to the next client who comes along. I suspect a lot of organizations are in the same boat: it is easy enough to move to java (and we are seeing that a lot), but you can't move away from perl in a modern unix environment (ok, within reason).

My question, then, for monks, is what skills you find yourself lacking, and would be receptive to getting training on, should some PHB at work hire a perl training consultant? I am very interested in hearing from both senior and more-advanced programmers, as well as the newer programmers who might now just be starting out as professional perl programmers.

Some ideas I had were:

  • introduction to functional programming
  • writing code for re-use
  • decoupling
  • introduction to OOP
  • using CPAN efficiently
  • basic editor configuration (i.e., using emacs and vim effectively)
  • common mistakes
What bothers me is where to start for the advanced users. I have met many advanced perl programmers who still dont use things like map, binary operators, tied hashes, who are afraid of globs, toying with references, multilevel datastructures, and so on. I'd also really like to work up a half-day DBI session since so many people are horribly mis-using it.

I'd really appreciate what feedback you can offer.

brother deprecated

Laziness, Impatience, Hubris, and Generosity.

Replies are listed 'Best First'.
Re: What training do YOU need?
by Aristotle (Chancellor) on Oct 20, 2002 at 18:09 UTC

    I believe CPAN should be introduced much sooner. You don't have to know how to write classes yourself in Perl to use object oriented modules, and various modules employ object orientation to varying degrees. You can start with simpler interfaces and gradually work your way to more completely object oriented modules.

    At this point, they can already work with highly improved efficiency.

    Then, you can introduce the features that make these work, one by one; starting with package to write simple libraries, and gradually adding nooks until you naturally reach bless. Because they've already seen how each of these crannies looks from the module user perspective, your pupils are likely to grok their purpose easier.

    Of course, the premise to all this is that they need to understand references. So that would be the first thing on your list; and logically, too, if you ask me, since any non-trivial datastructure - and therefor any well written code for a non-trivial task - requires them. Remember the foremost principle to good software design is good data structure design: the necessary code will nearly naturally follow from a well designed data structure. Arrays of arrays or hashes are much better than working with parallel arrays, an awfully ugly workaround for not having references. It is incredibly important to build this skill.

    So you have the following order:

    1. References
    2. Using modules, CPAN, using objects
    3. Writing modules, writing classes

    To me that looks like a natural way of going about it, and I believe it also offers the most practical value at any point in time. Everything your pupils learn will be immediately useful to them. Which means that they won't get bored with theorizing, so that it'll be easier to maintain concentration and build understanding. The bottom line is greater rewards for the pupils and, let's not forget, easier lessons for you and them.

    That's my thoughts after 15 minutes of thinking about it. :-)

    Makeshifts last the longest.

      I would just like to add, from a beginners point of view. To me that makes the most sences too. I think one of my biggest problems is looking at the big pictures and putting it all together. the lay out that you just gave seems to progress to the bigger picture and would help a beginner like me see how every thing fits together. I think that I wouldn't be able to fit all the pieces together right they way deprecated laid it out.

      learning Perl one statement at a time.
Re: What training do YOU need?
by gjb (Vicar) on Oct 20, 2002 at 17:44 UTC

    I think your list is quite good, although I'd not put functional programming on top. OOP is mandatory IMHO and some of the following you might consider adding to your list.

    I'd recommend starting with a good course on algorithms and (especially) data structures since one needs a good grasp of this subject for any sizeable programming task. Unfortunately I find this skill lacking even with more "advanced" programmers.

    Another nice skill to have for writing reusable code is patterns (Gamma et al.). Thinking in terms of patterns speeds up development and allows for better code maintenance.

    Some ideas coming from the Extreme Programming community are also worth familiarizing oneself with (first write a test, then write the code that will be tested; always do regression testing while adding code). I've done a few projects that way and besides being efficient, it was also satisfying.

    Just some ideas, -gjb-

      ++ to gjb's suggestion for test first coding.

      Using Test::More and friends in this way has improved my code and made writing it a far more pleasant experience.

Re: What training do YOU need?
by BrowserUk (Patriarch) on Oct 21, 2002 at 05:00 UTC

    It difficult to make an informed judgement on the basis of the description you provide, but from what you've said, I would warn you to be careful. As an 'external' who has been charged with 'bringing the internal guys up to scratch on X', I have met with some fairly hostile reactions to training that was perceived as being 'trivial', 'academic', 'theoretical' or as one guy put it, 'teaching your grandmother to suck eggs!.

    I had more success with a technique passed on by one of my mentors. Get them involved in the process from the start and make sure that they can see the relevance and benefit of the training.

    In this case, I would ask them individually or collectively depending on your judgement of which would be viewed in the best light, to nominate the "worst" piece of code in their daily/weekly routine.

    Ask them to provide you with a copy (minus any authorship info to save faces), along with sets of input and output data where applicable. Take this away, spend a weekend or week or whatever it takes to re-work it utilising the skills you wish to demonstrate. If you have time and the chosen piece lends itself to the task, do and retain multiple passes, concentrating on one aspect of your list at a time. Make sure that your final revision will operate as a replacement for what you started with, even if this stops short of best practice because achieving that requires modifications to stuff before and/or after in the processing chain.

    Your aim should be to show them how the concepts on your list can be used to real effect to improve their daily/weekly lot, and how they can be used to effect beneficial change within their operating environment. This is much less likely to provoke the kind of reactions that I listed earlier.

    If you can lay your hands on a good display projector, and make the changes to the chosen piece of code IRT, in front of them, rather than doing it as a set of slides or a PowerPoint (or unix eqiv.) presentation, so much the better. That also gives you the opportunity to impress the hell out of them with your power use of emacs/vim. Pick on a common example of code that can easily be improved using idiomatic constructs. Something like changing C-style for(;;)'s to foreach(@array) or map{..}@array. Make the first change manually explaining the reasons and benefits, then let loose a macro to convert the rest of them in the file, (even if you have to write the macro specially beforehand), and continue to discuss the benefits whilst the macro operates on screen.

    If you can find a good example of where a substantial piece of code can be easily and quickly replaced with a CPAN module, do it. If possible, do the installation of the module "before their very eyes". Crop the lump of code you are replacing and implement the replacement. Demonstrate that it works.

    If there is one thing I personally hate on training courses, and especially mandatory ones, is contrived examples that I cannot see how to apply to my real-life problems. I remember an OO course that was based around Yet Another Cutesy Animal Evolution Problem as a demonstration of Inheritance. Don't do it! If your audience cannot see the benefit of your examples to their real lives, it likely that your instruction will fall on deaf ears. Be really careful not to patronise or talk down to them, and I would avoid 'homework' tasks.

    On the other hand, if they see you reduce a large monolithic piece of their own code that they have already identified as a problem, to a set of nicely decoupled, potentially re-usable, easily maintainable and modifiable functions and/or modules, they are much more likely to accept their need for the instruction in the first place.

    Good luck. I do not envy you your task.

    Cor! Like yer ring! ... HALO dammit! ... 'Ave it yer way! Hal-lo, Mister la-de-da. ... Like yer ring!
      Good post!

      Pick on a common example of code that can easily be improved using idiomatic constructs. Something like changing C-style for(;;)'s to foreach(@array) or map{..}@array. Make the first change manually explaining the reasons and benefits,

      I realize you probably just quickly chose a this construct as an example. But your choice is unfortunate.

      Changing C-style loops to foreach loops is not as trivial as you would have it appear. The presentation would be impressive, but misleading.

      I might point out the advantages of foreach loops. And how the C-style loop is more cumbersome but more adaptable. These folk are probably very comfortable with this C phrasing. Saying this is how to improve the code is unnecessary optimization and quibbling over style at the risk of breakage.

      I like presentations that include workload and mental state, to wit:

      "It took me two hours to decipher this code."
      "Thirty minutes to get this rewrite, that has these problems. Notice how I used ..."
      "A hour of exploratory testing and minor tweaks. At this point I felt like I had a good understanding of ..."

      To the original post:

      If these are not fairly simple scripts and requirements are vague I'd first want my crew to instrument the old stuff to tee off both I and O on demand. Assuming this is reasonable.

      I cannot imagine the function-free java code these people are writing.

        But your choice is unfortunate.

        Interesting comment. Id be interested in hearing more about why you think this was an unfortunate choice.

        And how the C-style loop is more cumbersome but more adaptable.

        A C-Style for loop is always functionally and internally equivelent to a suitably constructed while loop. The primary differences being that many programmers from a C tradition find the for(;;) syntax more readable, while the rest of us find the while syntax more readble and maintainable. I am in the latter group. :-) Nine time out of ten (more really) when I see someone use a C-Style for loop they are not exploiting the equivelence to a while , but rather using a slightly less readble, more error prone and less efficient form of a for (LIST) loop. I avoid the term foreach because foreach is a synonym to for in perl, they are identical.

        E:\Smoke\source>perl -e "foreach (my $i=0;$i<10;$i++) { print $i,$/}" E:\Smoke\source>perl -e "for (my $i=0;$i<10;$i++) { print $i,$/}"
        Insofar that the majority of its use is to simulate for ($m..$n) where $m<$n I think advising people to convert to the more legible and less error prone form is probably a good idea. And I also think that its a good idea to use while () {} continue {} instead of for(;;) if only because it tends to be easier to read. Having said that in the time ive been using perl I think ive needed the construct a mere handful of times.

        Ultimately id argue that for(;;) is a construct mostly provided to reduce the pain for C junkies, and that in the long term better constructs that are more intuitive and less error prone are available and so should be encouraged.

        --- demerphq
        my friends call me, usually because I'm late....

Re: What training do YOU need?
by Juerd (Abbot) on Oct 20, 2002 at 22:14 UTC

    When you talk about CPAN, don't forget to focus on real-life situations. Too often people get carried away with Acme-modules and other not-very-useful modules. Teach your students how to recognize a good module. ::Simple modules are good (well, most of them), and often provide more than enough functionality.

    Your doesn't include style yet. Style is very important, and not always (certainly not in business environments) a personal choice. You could integrate this with the editor configuration section.

    What training do YOU need?

    I need to be trained in coding simpler. I use thinks like map, binary operators, tied hashes, globs, references and multi-level datastructures a lot, and it's confusing for beginning coders. And I should learn to comment more.

    Oh, and in your half-day DBI session, please mention DBIx::Simple once or twice ;)

    - Yes, I reinvent wheels.
    - Spam: Visit eurotraQ.

Re: What training do YOU need?
by dws (Chancellor) on Oct 20, 2002 at 23:04 UTC
    Basically, they have been maintaining software for the last few years. Very few of their applications use subs (some of these are thousands of lines long), and none of them use modules.

    I would start out by teaching them to refactor. To refactor with confidence, you'll need to teach them about Unit Tests. They can practice writing new code in unit tests, and gain confidence using subs and modules as they go about refactoring. When they come out the other end, they may have gained enough confidence to start writing new stuff.

Re: What training do YOU need?
by demerphq (Chancellor) on Oct 21, 2002 at 00:55 UTC
    Personally I think that if you gave them the task of creating a "standard" module for a relatively simple tied object, buildable with tests, implemented using a simple algorithm like a binary tree (or better a right threaded binary tree) then you would kill a whole lot of your birds with one stone.

    A tie is of course object oriented by definition, but with a very clearly defined interface with the added possibility of being able to use the item in a non tied pure OO fashion, or of adding supplemental utility methods.

    A self referential object like a threaded binary tree (and im thinking of a threaded tree, mostly because it makes FIRSTKEY, NEXTKEY (iterators) much simpler to implement without being too complex in terms of data structure, requiring only the addition of flag field to a node ) is obviously heavily referential (and you could make things interesting by getting half of them to use array based structures, and half to use hash based structures) and as it involves back references it exposes the idea of object destruction and scope effects.

    By building a module such as the above starting from h2xs, as specified in an easily accessable and readable module like Tie::Hash, having them write tests with Test::More and the h2xs/MakeMaker system they would learn the process that CPAN automates, learn how to build tests and installable modules, write pod, and generally improve the world. ;-)

    I guess if heavily referential structures werent to your taste, a reimplemented (hmm, i cant find it on CPAN) character array tie might do nicely (you know treat a scalar as an array of chars (but with more interesting semantics))


    updated: Couple of spelling and gramatical errors. It was late... :-)

    --- demerphq
    my friends call me, usually because I'm late....

Re: What training do YOU need?
by ignatz (Vicar) on Oct 20, 2002 at 22:44 UTC
    Besides testing the main thing that I would add would be relational database design. Just a hunch, but I'll bet that with code like that there have got to be some ugly database structures underneath it.
Re: What training do YOU need?
by sauoq (Abbot) on Oct 21, 2002 at 03:18 UTC
    I have met many advanced perl programmers who still dont use things like map, binary operators, tied hashes, who are afraid of globs, toying with references, multilevel datastructures, and so on.

    I have to question this statement. What makes a programmer "advanced" with regard to a language but familiarity and experience with all or most of a language's features? I would have a hard time judging a perl programmer as advanced if he didn't use map or binary operators and was "afraid" of globs but one that didn't use complex data structures wouldn't even be "intermediate" in my book.


    To answer your question though, I would be grateful for some training in Perl internals and Perl/C integration. I know both languages well. I've written a couple simple XS modules and I even managed to embed perl in another application once (though I chose the easiest possible way to do it.) Still, I have found my excursions into those areas to be slow going. I've basically bumbled through my attempts. An instructor would help immensely, I imagine.

    "My two cents aren't worth a dime.";
      I completely agree with your first point.

      And for your second recently there was a book released by one of the P5P gurus on perl internals. I cant think of the name off the top of my head, but I suspect you can track it down easily and that it would be quite useful to you.

      --- demerphq
      my friends call me, usually because I'm late....

Re: What training do YOU need?
by rir (Vicar) on Oct 21, 2002 at 03:26 UTC
    This sounds like salted ground when you say: advanced perl programmers who .. are afraid of globs, toying with references, multilevel datastructures. Especially if by globs you mean typeglobs.

    It sounds like they might hire one Perl toolsmith/mentor type.

    These are items that seem to come a bit slower to many programmers: tie, grep, map, BEGIN, .., ..., m, wantarray, scalar, and the special variables that likely want local and issues that go with all of these.

    The core modules are worth more attention than CPAN's.

    Things I'd like to know more about:

    • Filesystem organization of code for production & distribution.
    • Testing in the large.
    • Use of compile time constructs. Just curious.
    • The special vars and tokens.
    • Regexes.
    • The debugger. Just curious.
    • use fields, this seems in flux.
Re: What training do YOU need?
by rinceWind (Monsignor) on Oct 21, 2002 at 10:00 UTC
    deprecated, I could add some ideas to the many suggestions monks have offered you. However, I wonder if you have a bunch of mugs^H^H^H^Hfriends outside the workplace, to whom you can present any slides or course examples for comment before your colleagues see them.

    In my experience, a local Perl mongers group, if it has technical meetings, provides the ideal forum. Even if you don't give a talk, you can always scavenge for ideas while you chew the collective cud over a pint of beer.

    Also, you are quite likely to find advanced Perl programmers at a mongers gathering.

Re: What training do YOU need?
by lachoy (Parson) on Oct 21, 2002 at 13:00 UTC

    This is an excellent list. Since you're at this company you also have the benefit of being able to use real-life examples rather than the cookie cutter ones typically found in books. (With good reason, but still...)

    You don't say if this is your main job or not. If it's not, I can't imagine the company that's blowing $150K monthly on consultants would be averse to bringing in one of the many perl trainers (or training teams) for a week-long, full-time course. (See the PerlMongers training page for some companies, although the fact that merlyn is always here might tilt you toward Stonehenge :-) Training properly is hard, and these folks have tons of experience with situations like this. Plus they have time-tested materials that take a long time to get right.

    M-x auto-bs-mode

Re: What training do YOU need?
by rje (Deacon) on Oct 21, 2002 at 16:19 UTC
    On Topic:

    I would love to learn OOPerl in a classroom setting. I haven't spent the time to learn it well, and don't feel confident with it. For example, I couldn't just sit down right now and code up a perl class without having to refer to a concrete example.

    Functional programming would also be a very nice topic to be trained in.

    I guess my laziness is due to not needing those paradigms badly enough. In other words, "perl 4" can get the job done most of the time.

    Off-Topic: I really, really like the fourth perl programmer virtue. Thank you for including it!

Re: What training do YOU need?
by John M. Dlugosz (Monsignor) on Oct 21, 2002 at 17:02 UTC
    On your first item, are you sure you don't mean procedural programming (the use of subs, modules, while loops, etc.)? Functional programming is really strange and doesn't find its way into normal development, and is best left to the mathematicians.

    Update: handily accesses the Free On-line Dictionary of Computing. To summarize, functional languages are based on Typed Lambda Calculus, have no side effects, and examples include a subset of Lisp, Haskell, and Backus' FP Systems, which I used in my Masters studies at UTD. In that link, the author explains how functional programming is "very different" from procedural programming.

    A reply falls below the community's threshold of quality. You may see it by logging in.
Re: What training do YOU need?
by John M. Dlugosz (Monsignor) on Oct 21, 2002 at 17:07 UTC
    Hey, what about the rest of us? I'd love to hear why people are misusing DBI... too bad you're not writing a book chapter or online course so we could all join in!
      I'd love to hear why people are misusing DBI

      From previous experience...we had someone who wrote things like this:

      $sql = "select stuff from table where id = " . param('id');
      I said we should be using placeholders here for security and efficiency (we were using Oracle which can really benefit from placeholders) but this person said placeholders didn't work. After some investigation (on my part) as to why they "didn't work", I found out that the 'id' parameter had a carriage return on the end of it, so placeholders didn't work in this particular instance (though with some /^(\d+)/ de-tainting they did work), so this this person avoided them everywhere instead of trying to figure out why they sometimes "didn't work." For similar reasons, he also avoided arrays, hash arrays, and other common idioms, but since his code "worked", no one ever questioned it.