Finding the path

[Spidy]

Greetings, fellow monks.

I've been writing Perl for about 5 years now(which is sort of scary when I look back on it). I know how to use all of the generic programming tools like conditionals, loops, and variables - and I know how to use a few of the Perl-specific tools - like $_, dispatch tables, map and grep, and CPAN modules.

However, lately I feel like I've plateaued. I use references, complex data structures, and map and grep all the time - and I can't clearly see what I should be learning about now. Does anyone have any recommendations on what Perl has that I should learn, that will improve my code and help expand my skills? I primarily use Perl for general systems administration and web stuff/CGI.

Thanks,

Spidy

[moritz]

I noticed you didn't mention regexes. There's quite a lot you can learn about them. I strongly recommends Jeffrey Friedl's book "Mastering Regular Expressions" for that one. Nearly everyone can learn a bit (or a bit more ;-) from reading it.

Another book that I strongly recommend is "Higher Order Perl" by Mark Jason Dominus. It changed my way of thinking about programs and programming quite a bit.

Other nits:

• Read in perlmod about symbol tables
• If you don't already know it, read perlsec
• Read the unicode documents like perluniintro and perlunicode
• Go to the list of Saints in our Book, randomly click on a few of them, and look at some of their top 50 voted writeups. You'll find some real pearls there ;-)

[talexb]

Let me add one thing that no one's mentioned yet -- Test Driven Development.

At $work[-2] I developed a web application called File Exchange Server (FES) partially using TDD, and it opened my eyes to the whole process of a) starting with a known good system, b) adding a feature or bug fix and c) confirming that everything's still fine using your tests. Wow -- I change something over here (waving my hand over to stage left) and something over there (waving to the balcony) breaks! Cool! Now, why is that? Go find out.

My colleague Mike Stok turned me on to Log::Log4perl, a fantastically useful module for logging that allows you to turn up or turn down the logging level externally to whatever you're testing, with module granularity. Messages can go to a log file, they can be sent out as E-Mail messages, sent to a pipe, anything. Really, really useful.

As for what to build, try to build something you'd be interested in using, perhaps using wheels that exist already, perhaps using wheels that you write yourself. One thing that I want to do is sample my 200+ vinyl albums into MP3 files and make a jukebox out of the resultant recordings. I could use my wife's installation of iTunes, but it's a thought -- if I had the spare cycles.

A final word of advice -- keep your career in mind -- if software development is something you're interested in doing as a career, then when it comes to interviewing or even just talking about an opportunity, it's great to have some 'war stories' that you can roll out that embody your passion. That's your chance to talk about some of the cool projects that you completed, whether you used Perl or not.

Most of all, have fun!

[Your Mother]

Indeed. I recommend Perl Testing: A Developer's Notebook. It saved me at least 6 hours right off the bat with the section on testing untestable code, of which, to my dismay, I've inherited a metric oodle.

[chromatic]

Hah, that's exactly how I figured it out enough to write parts of that chapter. (Some of my metric oodle was in the Perl core.)

[cosmicperl]

Hi Spidy,
I had similar feeling recently. I ended up buying a load more perl books and joining some local perl groups. I soon found plenty of stuff I didn't know and could learn to improve my skills. I recommend checking out Perl Mongers, reading Intermediate Perl and Mastering Perl, and looking at contributing to CPAN.

There is of course going back to C and doing some XS perl.

Lyle

[papermonkey]

Hi!
I also recommend Perl Mongers, when done reading or browsing this great book, get Perl Best Practices, You won't regret it.
Otherwise, Mastering Algorithms With Perl has always been one of my favorites.

Paper Monkey

[Your Mother]

What everyone else is saying. Getting involved in a dev group can be a big growth catalyst too. Peer pressure and publicly viewed code (the code we write for others tends to be a lot better than that we write for our own projects; for me anyway) is a great way to learn. Moose community? Catalyst? Rose? DBIC?

Another excellent tactic for learning is to switch gears entirely. Try Ruby, Java, electronics projects... I found that working with Ajax and JavaScript changed my understanding of CGI/HTTP pretty dramatically even though I'd been doing CGI for years. Rooted out all the weak spots in my competencies. Slinging X(HT)?ML only became natural to me after learning to sling the DOM from JS.

After picking up any new technology you will find your Perl view improves. There are many interesting problem spaces being actively worked on. Social network stuff, distributed data, de-centralized security...

Another big winner is to pick up an abandoned CPAN module. Or start a new one for something like a new web service no one has had time to do yet. Forces you to learn deeper topics like testing and distribution.

Diving into the books can be nice -- I own just about all of them; some in multiple editions at this point -- but I find that I only absorb the stuff well when I can apply it to a problem space.

[Anonymous Monk]

No, don't try Ruby and Java. Ruby is the same language as Perl. Java is not worth learning (there are no concepts in there that you don't already know). Learn Lisp, Haskell, Smalltalk, OCaml, etc. instead. Those languages each contain different concepts that you can make use of in Perl, but might not already know about.

[Your Mother]

I would like to agree. Better but each of those has a pretty high barrier to self-edification and fewer places it will be used in a regular career. Plus, some of what you're talking about in "different concepts" can be handled with different code style in Perl. That's why the Higher Order Perl is so nice, it covers some of that.

I tried to teach myself some Lisp about 3 years ago when I was feeling the same, I'm not progressing feeling, and the resources, tutorials, etc are slim/difficult for a semi-hacker. If I had time now and wasn't resume padding I'd play with something like one of the nascent Parrot driven languages like Amber; or maybe Erlang.

[footpad]

Personally, I find I learn more when I use tools to solve problems I haven't tackled before. Perhaps you should look at the work you do and the tools you use to do it. Perhaps you'll identify a problem you can tackle.

Barring that, perhaps you can think of a project that you have wanted to attempt and then give it a whirl.

Perhaps you can think of a module you've always wanted to have.

Perhaps there's a project you've always wanted to redo.

Inspiration can come from many places; you just have to be open to hearing the voice urging you to take a crack at something.

As always, YMMV...

--f

[johngg]

Another topic you haven't mentioned is object oriented programming. Perhaps you could explore writing a module that provides an OO interface to something you use in your day-to-day work. I enjoyed reading TheDamian's Object Oriented Perl which gives an easily accessible introduction to the concepts before taking you deeper.

Cheers,

JohnGG

[dvryaboy]

Some ideas:
• as far as generic tools you didn't list -- pack and unpack are extremely powerful. You might already know them, but worth mentioning.
• learn to use templates (Template::Toolkit is a good one).
• try writing your own module, with unit tests -- use http://www.perlmonks.org/index.pl?node_id=158999 as a guide.
• What's this Moose thing all about?
• Read some of the Meditaions (example: reading and modifying bitmaps in pure perl)
• stop writing perl scripts, start writing perl programs.

[sundialsvc4]

My answer would be:   it's not about the Perl.

Look for a broader range of “things to do with a computer,” no matter what kind of thing it is and no matter what tool(s) you use to do it.

In the big-picture of software development, “writing the code” is really the very last thing that happens, and in many ways it is the least-important. Plenty of basically-talented programmers fall into the very same hole of being stuck with cleverly-written but non-managed projects that have become an expensive albatross around the neck-bone of anyone who touches it. (I can tell you from very-recent experience that there's nothing worse than trying to deal with a mass of 11-year old code that was all thrown-together at 2:00 in the morning. Especially when those same people who first wrote it are going to be slugging away at 2:00 in the morning tonight to meet yet-another thrice-missed deadline. Some folks never learn...)

In the big picture, you not only have to “write code,” but you also have to write maintainable code, and to maintain existing code, and to meet-or-beat every deadline and every promise that you ever make. While finishing your workday by six o'clock. (P.M., that is.) This is about learning how to use Perl well.

In time you may decide that your interest in code-pushing is waning and that your own view is naturally expanding to a “big picture view.” Go with it. Project management is a big-money skill. Seriously, it helps to work your way into a profession that you can easily discuss at dinner parties, with people who don't wear pocket-protectors to dinner parties...   :-)

[Spidy]

Thanks for the responses, everyone. They bring to mind some things I should go about learning, and some things I should clarify.

First off, things I should learn - I'll be picking up Mastering Regular Expressions and Higher Order Perl fairly quickly.

Secondly, a clarification - right now, I don't actually work with Perl for my dayjob(or at least, not most of the time - our sysadmin gets me to write him tools every once in a while) - my employer is running a pure M$ shop, and so I work primarily with ASP.Net and C# for my day job - all of my own personal projects are Perl, however.

I know a few other languages, in addition to the standard Web Developer pile - Javascript, XHTML, CSS, and at least one server-side language(in my case Perl, although I can fumble my way along with PHP as well).

I actually already own a copy of Object Oriented Perl - I thought it was interesting(as was Moose), but I haven't encountered a situation where it immediately sprang to mind that "hey, I should use OOP for this!" - I generally design my projects in such a way as to not need OOP. I do use it for some things - I just don't encounter too many situations where I'd like to use OOP and don't already.

I guess, really, I'm looking for cool things I can learn that I don't already know that can speed up my development time and let me reduce repetition. That's why learning about things like map and grep was awesome - because I could instantly see how much they saved on time(and keystrokes).

[GrandFather]

For small scripts (say < 100 lines) OOP generally doesn't matter much, but for anything much over that it can provide some really useful leverage. Even poor man's OOP like:

use strict;
use warnings;

my $obj = bless {hi => "Hello world"};

$obj->greet ();


sub greet {
    my $self = shift;
    
    print "$self->{hi}\n";
}

can end up saving a lot of time and confusion by avoiding passing lists of unnamed parameters into subs or accidentally doing weird stuff with global variables.

[doom]

OOP is one of those things that was clearly over-sold and is far too easy to use wrong (like, if you're doing elaborate designs relying on inheritance, you're almost certainly doing it wrong); but I've come around to using it by default even on relatively simple projects, largely because it's a widely accepted compromise between the wide-open style of using globals all over the place, and the tight, purely functional style of passing everything in and out explicitly.

[adamk]

Absolutely without a doubt the biggest step up I ever made was to learn how to write CPAN modules myself, not just use other peoples.

You learn so much from it. Writing clear and effective docs, testing properly, component-based development, dependency issues, cross platform development.

It's like the difference between being able to build things with lego, and being able to create your own lego pieces.

Once you get used to it, you'll find that you start building work projects as CPAN-style distributions as well, and this can bring huge wins in terms of maintenance and reusability.

And this drives an understanding of not just programming in Perl, but in creating large applications and large systems in Perl. And the lessons from that drive understanding in creating large applications and large systems in general.

[reyjrar]

A lot of good suggestions already. I'd like to add and reinforce a few.
• Maybe check out a YAPC if you haven't? A _LOT_ of fun.
• ORM's (DBIC,Rose,etc) are interesting and worth learning.
• Event Driven programming with POE is certainly a worthwhile excursion. All of my monitoring scripts and various daemons are POE based now.
• MVC Frameworks! (Catalyst/Jifty)
• If you're doing web stuff, why not pick up a JavaScript framework or two? I've heard and read good things about the following: ExtJS, jQuery, Dojo, and YUI.
Just a a few ideas. I'm actually still playing with that list. There's years of learning there.

[aufflick]

Higher Order Perl: ISBN 1558607013

[andreas1234567]

Risking to echo footpad, I suggest you use Perl to solve a real-world problem that interests you. Let me share my experience:

I became aware of the concept Local Shared Object (aka Adobe Flash Cookie) while listening to a podcast. There were privacy issues involved that triggered my attention, and since there were (at the time) no appropriate modules on CPAN, I ended up writing one that I released on CPAN.

As adamk, I have learned a lot by releasing code publicly, including

• proper testing
• version management
• module dependency management
• platform differences (e.g. endianness)
• supporting multiple versions of Perl

[nothingmuch]

http://swiss.csail.mit.edu/classes/6.001/abelson-sussman-lectures/

[Your Mother]

Nice. They've put a lot of ongoing lectures up as podcasts now too: Podcasts at MIT. Don't know about the CS stuff, or if any is even available, but some of the Physics stuff is great (only seen part of one... need that 30 hour day).

[nothingmuch]

These specific lectures have been some of the most mind opening CS material I've ever come across. Highly reccomended =)