Seeing the Forest for the Trees

[Dragonfly]

Occasionally during my day, I find that I have gotten caught up in some programming detail, and that mysteriously the day has gone by, and the deadline for the project that I am working on has slipped yet again.

I always have a tendency to feel self-righteous at this point, thinking “Well, I need to know exactly how this works, or there may be an unintended consequence in my code implementation,” and I feel not only this need for justification, but also a bit embarrassed for not meeting the deadline.

Having to ask for more time to complete the part of the project for which I am responsible (or irresponsible, as the case may be ;-) is a very stressful situation, and I have learned the hard way that it can easily lead to exhaustion, which tends to lead to even more lost time stemming from the extra bugs in my code base.

I know a great many programmers who are always looking for new ways to do things, and over my admittedly short and not-so-illustrious career as a web programmer, I’ve found that one of the hardest things to learn as a programmer is the Gestalt Theory, or as I like to call it, learning to see the Big Picture.

In this age of program complexity, there are a multitude of different areas in which one could conceivably get stuck, and without some view of the overall project, you can spend whole days, weeks, months, or even years studying just a single area or aspect of programming. In some scenarios this is fine; for instance, if you are a GUI specialist working on Aqua for Apple, for instance, then it is your job to study user interfaces, and to know them better than anyone else. And if you’re working as a DBA for a multinational banking conglomerate, well, who cares if you don’t ever bother to learn mod_perl? (I know, sad but true!)

Just offhand, I can list a dozen programming areas; sockets, file I/O and permissions, graphical interfaces, DBI, persistence, multi-threading, even deceptively simple areas like table design in SQL databases, in which I could conceivably spend weeks mastering the subject matter. This is a luxury I can’t afford, and I know of only a very few programmers who have found the means to study the areas that interest them at will.

With such a large number of programming aspects available to us, how can one hope to become an expert in all of them? Well, the short answer is that you can’t. Although I would absolutely love to spend a whole day playing with Astro::Moonphase, I simply cannot afford to do so while working on a project. Eventually, one must choose, and specialize in those areas we wish to master. Eventually, we learn to trust the skills of those around us; to rely on the works of others to take care of the nitty-gritty details of those areas in which we are perhaps not so skilled.

And that’s why Perl is such a marvelous beast. Because Perl, to some degree, allows me to select a combination of parts built by some very skillful engineers, without necessarily having to know how to run a tool & die shop myself. Perl lets me see the forest for the trees, as the expression goes. I sometimes feel like CPAN is a giant machine shop, filled with expertly forged axels, shiny spark plugs, and CV joints, and it is my job as the mechanic to figure out how to build a working engine.

When we put those parts together into the engines that drive our sites, we might do well to appreciate the skill that went in those carefully turned modules, but also, perhaps even more importantly, realize that if we had to do it all ourselves, we would never have the time to spend learning the fun stuff. ;-)

[nysus]

I've been self-studying web technologies for only about the past year or so. But I still get the sense that you are correct to say that it is impossible to master all the different technologies out there, especially since they are continually evolving. Even if you were able to master each area, you would soon forget after about 1 month of disuse. What I think is possible is to at least get a rough idea of the structure of each major technology/field. Also important to know is the key books, references, sites, etc. that you can refer to quickly look up the specifics.

Personally, I have found that always looking at the forest discourages me. I am actually more productive if I focus on the minutest details and forget about the big picture (of course, I am not on a deadline...this is all a hobby for me so far). I love taking the time to focus on one leaf on one branch of one tree and getting very familiar with it. I have learned quite a bit in one year using this approach. I think after 5 years I will have covered a good size chunk of the forest. Above all, though, the most important thing is to have fun exploring it.

[footpad]

Eventually, one must choose, and specialize in those areas we wish to master. Eventually, we learn to trust the skills of those around us; to rely on the works of others to take care of the nitty-gritty details of those areas in which we are perhaps not so skilled.

Yes, and if you do not choose for yourself, your projects will choose your specializations for you. If you are not very careful, you may find yourself specializing in things you really don't enjoy or care about.

Some solutions, I think are:

• Keep an active eye on the progress of your technical skills and the ones you wish to develop. If you cannot do that in your current situation, then you need to choose between the safety of the known and the risk of the unknown.

• Find ways of networking with skilled peers in the fields you wish to cultivate. This will help you build your technical skills and it will help you learn of opportunities more aligned with your desires.

• Invest in your technical skills on a regular basis, whether taking a few hours over the weekend, attending local user groups, lurking in appropriate technical communities, or whatever.

• Use personal projects to learn the basics of the skill you want to grow. This will help you find time to actually build them. By creating things you care about, you'll be able to maintain your enthusiasm while crawling up the learning curve.

• Prioritize your To Do list and manage that like you'd manage any other project. Give yourself schedules, task lists, and requirements. This will help you develop and/or maintain your design discipline.

• Never stop asking, "How can I do this better?"

The Perl community is really quite fortunate to have such a broad range of talent willing to help others learn. Between the O'Reilly books, CPAN, and sites like the Monastery, it's very easy to find new and better ways of doing things.

I've only seen that type of teamwork in a few other communities and none of had the breadth of support that the Perl community does. If this is important to you, then I suggest you use it as much as you can.

--f

[dws]

One of the double-edged swords that comes with maturity is the realization that you can't do it all. To make progress on the things that are really important to us, we have to first get clear on what they are, and then make the hard decision to scratch from the list some things that aren't important. Of necessity, some fun stuff goes by the wayside.

It's easy to be envious of someone who does seem to have time for all of the fun things we don't have time for. It's harder to see that they, too, had to make choices.

To make time for fun stuff, the wiser among us look for leverage. CPAN can be a lever. Humility can also be a lever. Humility allows us to seek help from others before we've spent 10 straight hours of frustrated debugging trying to get our code working to meet a deadline.