Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change

Right tool for the job?

by chester (Hermit)
on Sep 24, 2005 at 17:30 UTC ( #494784=perlmeditation: print w/replies, xml ) Need Help??

Hello Monks,

In an ideal world, I would know 3,000 programming languages. And when faced with any task, I would open my Rolodex o'Languages (with all of which I am quite proficient), and merrily flip through it to find the perfect tool to solve my current problem, mumbling to myself "Ada? no... ALGOL? no... Applescript? no... ASM? no... AWK? no..."

Half an hour later, I would leap from my chair, exclaiming "The right tool for this job is a combination of UNISAP and PACTOLUS, with just a touch of ASDIMPL and XPOP!" And it would be the perfect tool for this job, and I would whip up an elegant, simple bit of code which efficiently solves my problem while being readable and easy to maintain.

In the real world, I'm somewhat limited by the number of languages I know, and I'm limited by the amount of time I have to dedicate to learning new languages.

For this reason I sometimes tend to use Perl in perhaps inappropriate ways. For example, I have never learned bash scripting beyond the very basics, or sed, or awk; when I want to do anything non-basic involving the shell, I pipe it through Perl. I needed to write a quick GTK app one day, so I wrote it in GTK2-Perl. Another day a QT app, again I used Perl bindings. When I need to create an Excel document with a report from MS SQL-Server, I shudder a bit, then use Perl. When I use vim, I avoid vim-style regexes and use perldo. And so on, and so forth.

Often I'm pretty sure there's another language or tool which was made to solve the problem I'm solving. Instead I wield Perl like a Neanderthal and bludgeon the problem to death, because I (unfortunately?) have a tool which can do anything I want, and because Perl is What I Know. And it works, for certain values of "works". If I was writing an OS kernel or a control system for a nuclear reactor, of course I wouldn't use Perl, but short of that, Perl can solve almost every problem I want to solve.

To put it another way, I think there are short-term and long-term costs. In the short term, using Perl all over the place is good because I can solve problems using resources I already have. (Knowledge being the resource in question here.) But if I spent the time to learn Java or Ruby or assembler language (or whatever) really well, it may save me time in the long run, by avoiding Perl problems that arise when Perl isn't the ideal tool.

But I know that if I wanted, I could devote the next decade to learning more and more about Perl, and still never learn everything there is to know. I still learn something new every single time I come to PerlMonks; I don't see that stopping any time soon. And much of what I learn about Perl is applicable to other languages. But much is also not at all applicable to any other language; it's learning Perl, and it's useful if I use Perl, and it's a waste of time if I don't use Perl.

My point (there is one, I assure you!) is this: Is there a point where it's time to say "I know Perl 'well enough'; it's time to devote most of my resources to learning something else"? If only for the sake of having more tools available? Do you think it's more important to devote one's time to becoming a jack of all trades or to try to become a master at one thing in particular? Assuming one lacks the time or resources to do both, or assuming that one is looking to decide where to allocate the most effort.

(update: Thank you all for the thoughtful replies.)

Replies are listed 'Best First'.
Re: Right tool for the job?
by GrandFather (Sage) on Sep 24, 2005 at 19:54 UTC

    There are two things, the right language, and the right library. Often the library available to perform a task dictates the language.

    I think also that your roladex can be a lot smaller. Many languages have been subsumed by later ones. There are very few places where you would choose Algol over C++ or Basic over Perl for example. There are many many languages, a lot of which have been created for a very narrow range of application.

    In practice knowing a good and well supported scripting language (Perl?) and a good and well supported general purpose language (C++?) and the "languages" associated with particular problem domains that you are interested in (MatLab, HTML, ...) will get you where you want to go in pretty short order. By well supported I mean that you should find the language available most places that you go and that there is a lot of good third party code available in librarys of various sorts.

    Appart from anything else, knowing a small, but diverse set of languages well will make it so much easier to learn and use anything else that you may be forced to pick up for a particular project, that in practice your half dozen language roladex actually encompases all the languages that you are likely to need.

    Perl is Huffman encoded by design.
Re: Right tool for the job?
by Tanktalus (Canon) on Sep 24, 2005 at 20:14 UTC

    It's all a matter of priorities. Thus, I'll respond from my own priority list.

    Top of my priority list is family and friends. Generally in that order. Programming, though a very fun hobby, and a reasonably lucrative way to earn a living, is not at the top of the list.

    Now that I've established that I have other very important things to do with my time, now I need to allocate what time is left for career purposes. Top of this list is to know what tools are available to me. Note that I did not say "know the tools" but "know what tools are available." I don't need to be a Java expert. I do, however, need to know what Java's strengths and weaknesses are. Ada, as powerful of a language though it may be, doesn't make the list because in my work environment, Ada isn't available. Nor would any Ada software be maintainable because there probably is very little Ada knowledge where I work. Better to use a language that we can find others to come in and work with.

    From there, I've decided (or, in some cases, someone had decided for me) to learn certain languages. C/C++ I knew before graduating. Shell, awk, sed, (including as little C shell as I could get away with) perl, and even to some degree Java, I learned after joining my current employer. Perl is what I've mostly stuck with simply because it's the most enjoyable way to gain the most productivity.

    That said, I highly encourage you to learn more shell. It's an awful language. But there are some really cool utilities, such as awk, sed, grep, from which you can learn other ways of approaching a problem which you can take into perl to get something that runs inside the same process as the rest of your code. Also, there are some things that just are useful to know - quoting rules for running perl one-liners, other shell keywords and special characters, all of which are important to know when using system. I've found that knowing shell way better than I wanted to has helped me take advantage of perl - and learning perl has greatly helped me in the way I approach shell problems.

    Other languages, such as C/C++, Java, C#, BASIC, ... these are primarily for other areas of programming from this. The above is helpful for system administration. C, C++ have strengths in bare-metal computing speed, but are relatively weak on TCP and GUI programming. Java is good for TCP and GUI programming, but not so good at setup costs (i.e., bringing up the JVM is expensive) which is one of the reasons why Java is used for servlets - where they aren't taken down. C#, BASIC - I'm not so familiar with these as I've managed to mostly avoid Windows (and our team doesn't use C#, and uses just tiny bits of Visual BASIC). So, knowing what is best for each aspect of our project, I am in a reasonable position to parcel out the requirements to the teammembers who actually are most proficient in the respective tools. (Although I think I'm still the most proficient in C++, I don't actually get any of those assignments.)

    I think one issue for you may be one of timing. Technical management is supposed to have a handle on many things below them, but need not be deep knowledge of any area. So, if that's the career path you want, you need to prove yourself as a master of your area, and then, as you work your way up the chain, you gain a broader knowledge of more things, but don't actually need a deep knowledge of anything, all while getting paid more and more. Theoretically ;-) You just need to become the master of one thing in particular until you get those promotions.

    But don't sacrafice anything that's actually important over this. And I'm not talking about the lip-service that most people give, "Oh, of course my wife/husband is important to me. Of course my kid(s) are important to me." Based on your actual actions, what is actually important - are you living up to your priorities? If family is important, are you spending more time with them than your career? Sometimes quantity is more important than quality.

      Thank you for the reply. Semi-off-topic: I agree about family and friends being priority, but work is sometimes a priority when it comes to feeding said family or being able to spend time with said friends. Work is a priority insofar as it enables a person to do what's really important.

      I know enough shell to properly quote a Perl one-liner or mess with the environment or pipe things around, but Bash's loop syntax (for example) is beyond me. I'm glad to know I'm not missing much.

      I'm not sure about management, but I'll keep it in mind. I'm blessed with a manager who says "I know nothing about computers. Do your own thing, just get your work done and I'll be happy". Seems like that isn't the norm. So I have the freedom to learn and use anything I want. Which may be part of my problem, this being my first programming job after college. At the same time I love it, of course.

        Bash's loop syntax is the same as Bourne shell's loop syntax. But I don't think that's the point of your observation ;-)

        To be completely honest, if this is your first programming job after college, I'll give you a piece of advice. Of course, you have to remember that it may be worth precisely what you're paying for it. The advice is this: it is often a good use of your time to get a job where there is a senior expert to learn from, at least at first. My first co-op job was in a department where the manager was like yours - didn't know much about programming or anything, just wanted me to do what I had to to complete the job. (Oddly, that was my ForTran to C job.) The only other software developers around were in another department which had a strained relationship with mine, and made it extremely difficult for me to learn anything about software development.

        My second co-op job was in a software development company - they made money from selling software licenses. So I had a software development team lead, a former software developer as a manager, and a current-developer as a business unit manager. I had some really good influence from them, and learned a lot.

        Then I graduated and started my current job (8 years ago already). And again with the being surrounded by software development. However, I think I now have the experience to go solo in another company if I wanted, and I'd feel comfortable that I was efficiently solving their software problems.

        Best of luck :-)

Re: Right tool for the job?
by perrin (Chancellor) on Sep 24, 2005 at 20:01 UTC
    That old "right tool" thing usually gets dragged out as a way of saying "I don't want to use the tool you like." Perl is very good at a much wider range of tasks than most languages, and I don't see much point in learning bash programming or vim regexes when you already have a tool that does a great job.

    Learn a new tool when you want to do something that Perl is lousy at, like CPU-intensive number crunching or writing device drivers. Or learn a new tool because you want exposure to new ideas (there is a lot of great writing about software development that is centered around Java or C). But don't feel bad about getting things done with tools that you know.

Re: Right tool for the job?
by sauoq (Abbot) on Sep 24, 2005 at 22:33 UTC

    Often enough, the right tool is the one you know because you know it. It doesn't much matter whether you use a slide rule or a TI-89 Titanium to solve an arithmetic problem so long as you get the right answer.

    Of course, there are problems where the tool does matter (which you are well aware of as evidenced by your assertion that you wouldn't use Perl to write an OS kernel.) But they usually aren't that hard to recognize. In the event that they are, and you use Perl but you get slaughtered on performance, all is not necessarily lost anyway. You might have a good prototype.

    The things you listed that you've used Perl for seem to me to be fine uses of Perl, though. Perl is great because it's like a Swiss Army Power Tool. It not only saves you time and effort with the software you write but, it saves you the time and effort of learning those 3000 languages.

    If it was Prolog you knew instead of Perl, what would you do in vim? Wait for someone to write prologdo? Ever seen QT bindings for awk? Think you could create an Excel report from data in an MS SQL-Server DB using csh?

    I'm sure you get the point.

    If you really feel the need to learn another language, by all means, go do it! The broad base of development experience is certainly a good thing. Not so much because it helps to know a bunch of languages, but because you tend to learn different ways of looking at problems. But don't be too concerned about relying on what you know. Depth of development experience can be a good thing too. The more things you use Perl for and the better you get with it, the easier it will be for you to recognize its limitations and know when it really is the wrong tool.¹

    [1] Which, of course, is almost never. ;-)

    "My two cents aren't worth a dime.";
      Often enough, the right tool is the one you know because you know it.
      Unfortunately, there's often other considerations ... think about maintenance -- if for example you're asked to add feature X to a client's IIS/ASP web site, you probably don't want to choose perl (and TT or whatever) (or mysql/postgres vs their existing SQLServer) even though it might be very easy and fast to write, because they won't have any ability to maintain it and it will take a decent amount of installation/configuration to put those tools in place to begin with. (and yes, you spend the whole time cursing vbscript, but it's about the client & solution, not being able to always use the most fun language)

      Ever seen QT bindings for awk? Think you could create an Excel report from data in an MS SQL-Server DB using csh?
      Those are dangerous questions w/this audience -- someone is liable to go do it for fun ;)
        Unfortunately, there's often other considerations ...

        Well, yes... which is why I went on to say: "Of course, there are problems where the tool does matter (which you are well aware of as evidenced by your assertion that you wouldn't use Perl to write an OS kernel.) But they usually aren't that hard to recognize."

        I didn't mean to imply that performance was the only other consideration, by the way. Or even that the only considerations are technical. More often than not it comes down to politics and culture.

        "My two cents aren't worth a dime.";
Re: Right tool for the job?
by Anonymous Monk on Sep 25, 2005 at 00:53 UTC
    Leaving aside domain specific languages (for instance SQL or TeX), I've been able to do my jobs for the past 20 years using Perl, C, shell (+ standard toolset, including awk). I only program in other languages than Perl, C or shell if it's a matter of modifying existing code.

    Perl is my first choice when writing something new, but I use C if either the programs running time is important and Perl is too slow (that's seldomly the case), or when I need to do something that's either quite bare-bones and/or system specific. And I use shell (or C) if I need to code something for a platform where Perl isn't available. Part of my job involves writing code for small machines, that neither have the disk space to install Perl, nor have the memory to run it.

    That doesn't mean other languages couldn't do the job. Java, Python, .NET, Ada, whatever, they all could replace Perl easily. But the combination Perl+C+shell does a complete coverage of the solution space I've encountered. But where I can imagine Perl being replaced - I could not fanthom to be "complete" without C, or without the shell.

Re: Right tool for the job?
by Zero_Flop (Pilgrim) on Sep 25, 2005 at 07:20 UTC
    One thing to consider when selecting the "right tool" is how well you know that language. That will have alot of impact on what you are doing. For example let's say you have a problem and technically C++ would be the best language to use. But you know perl better, and will be able to produce high quality code in less time if you code it in Perl. Then maybe for you Perl is the right tool.

    When selecting the right tool you have to consider the resources and not just what a particular Language brings to the table. If no one knows java in your company, then why code something in java, even if it is the best language for that applicaion.
Re: Right tool for the job?
by zshzn (Hermit) on Sep 25, 2005 at 01:56 UTC
    Perl is a great and diverse tool. It is not always the right tool for the job, nor is it always the best tool when it does the job well. Learning other languages not only increases your available options, but also your understanding of technical concepts, and ultimately your understanding of Perl. Learning is a good thing. However, your capacity for learning is often limited by your free time, and choices have to be made. I would advocate a balance, learn more languages while keeping up with Perl. Not 3000 languages, of course. A useful list of popular languages that you may need to work with includes Perl, C, ASM, shell, Java, PHP, and Python, and the list just goes from there. Certainly not 3000 useful languages, you could get by very well with half a dozen.

    If you don't know C, any ASM, or shell scripting, I would suggest an attempt at learning the basics of them before going further with Perl.

      A useful list of popular languages that you may need to work with includes Perl, C, ASM, shell, Java, PHP, and Python
      While learning a popular language may make you more popular, learning an unpopular language will probably be more beneficial in terms gaining programming wisdom. Choose one from each of the following categories...
        Certainly. However, he mentions programming languages as tools to solve particular tasks that may come up, not so much in a 'pure' kind of interest for the sake of learning. Although any of those would be very beneficial to a programmer's knowledge of concepts, learning a few popular languages will not only teach new concepts and wisdom, but also will be very practical.
Re: Right tool for the job?
by Anonymous Monk on Sep 26, 2005 at 19:31 UTC
    In an ideal world, I would know 3,000 programming languages. And when faced with any task, I would open my Rolodex o'Languages (with all of which I am quite proficient), and merrily flip through it to find the perfect tool to solve my current problem, mumbling to myself "Ada? no... ALGOL? no... Applescript? no... ASM? no... AWK? no..." Half an hour later, I would leap from my chair, exclaiming "The right tool for this job is a combination of UNISAP and PACTOLUS, with just a touch of ASDIMPL and XPOP!" And it would be the perfect tool for this job, and I would whip up an elegant, simple bit of code which efficiently solves my problem while being readable and easy to maintain.

    In an ideal world, you'ld know only one language; one that was expressive, powerful, high-level, and easy to read. Everyone would use it. Performance wouldn't be an issue, because the language would have a mythical self-optimizing compiler that could generate faster code than the best hand-tuned assembly. It would determine intent, and re-write your algorithms to equivalent, faster algorithms on the fly. It would sort lists in constant time, chop onions, and make jullianne fries.

    While you're dreaming, dream big! :-)

      In an ideal world, you'd know only one language...

      Hey, isn't your list the planned feature list for Perl 7?

        Hey, isn't your list the planned feature list for Perl 7? Almost. Perl 7 won't make fries, and cutting onions is not the perl way. :-)
Re: Right tool for the job?
by samizdat (Vicar) on Sep 28, 2005 at 16:17 UTC
    Since this question has been visited so many times from the language-to-language and programming paradigm comparison perspectives, I'll see if I can expand it in a slightly different direction. As much as the variety of coding styles (functional, logic, OO, pure recursive, etc.), programming languages have different working environments. Besides gaining knowledge of different syntax and supplied constructs (RUBY for OO, for example, or SNOBOL for pattern matching), it's also good to look at different modes of programming.

    First, a good subject to explore is programming a microcontroller in assembly language. Some micros have easier programmers to use than others, like the PIC from GI and the AVR with bootstrapping FLASH from Atmel. If you can get your hands on an Atmel Butterfly demo board, you can do an incredible amount of experimenting in both hardware and firmware directions. (Available from DigiKey in single quantities for around $20.) These single-chip systems allow you to learn to appreciate the true cost of advanced programming constructs as you interleave interrupt handlers with main line code. They are so primitive, yet so powerful, that there's much to be learned there.

    The whole concept of a UNIX shell is so powerful, regardless of what you might think of BASH or csh syntax, that it's well worth studying as a concrete example of interpretation at its most powerful level. Studying other parts of your OS source code, such as the scheduler, TCP/IP stack, and various daemons, are also worthwhile for your growth. Perl can be used to do various tasks, and is so used, in many UNIX systems.

    Different forms of IDE and framework are also worth studying. Smalltalk has already been mentioned, and I really miss APL as the best example of an interactive calculator-style interface. I once used a really awesome window builder IDE built on top of Smalltalk (called Window Builder) that you could use to do most of a GUI-based OO app with virtually no coding except for your various extensions to the Collection classes that provide your app's data structures. Another example (though less effectively implemented) is the Microsoft VB programming system with its forms and other constructs. Many of the concepts embedded in these kinds of programming environments can enhance your programming capabilities by providing nuggets of insight.

    Another form of programming to study is compiling. Not compiled programs written in C or some other set of syntax, but the compiler / translators themselves. The concepts of declaring your syntax, parsing it, creating symbol tables, etc., are full of concepts that are fundamental tools in programming. Most books in this area are dense and arcane, but you can get a lot from studying the manuals and tutorials for things like lex and yacc and Backus-Naur descriptions of the syntax of various languages.

    A final area of study that I'll mention is the area of knowledge representation and pattern matching. There are many ways of approaching this (from BNF to Perl RegEx to expert system rule bases), but the fundamental concepts can help you build more readable and maintainable programs. A key concept I use in many of my programs is to study the states of a system and the rules for transitioning from one state to another. By creating rule sets and a problem-specific set of declarative and action constructs, one can create a 'little language' that makes the problem domain explicit and concrete and easily extended.

    By understanding more of how your computer's software is layered and interleaved, you can make your own additions much more efficient and understandable. Many current CS courses of study that I've looked at do not make enough of an effort in this direction so that students learn to understand the methodology of embedding their program into the real machine and software system that it will run on. While it is important to understand programming abstractions thoroughly, understanding the underlying systems and plusses and minuses of various programming methodologies is crucial to making effective use of your machine to solve your problem in the best way possible.
Re: Right tool for the job?
by deep submerge (Scribe) on Sep 26, 2005 at 16:40 UTC

    I'm tempted to throw out the metric that... Whatever programming language can accomplish your job in the least amount of code... Would be the one to choose. With "least amount of code" as a factor in being the most productive...

    (I wish I thought this out better... No wonder I don't post many write-ups...)

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://494784]
Approved by gmax
Front-paged by itub
[karlgoethebier]: Good morning
[Discipulus]: good morning vietnam!
[Discipulus]: but there is some perl workshop somewhere? I notice many high level monks disappeared.. we can make PM a squat in the meantime ;=)
[karlgoethebier]: Discipulus: Mmh, who disappeared?
[marto]: morning all
[shmem]: perl -lE '$_="odd mornin";($o=$_)=~ s/^|$/g/g;say$o'

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (7)
As of 2017-06-27 08:05 GMT
Find Nodes?
    Voting Booth?
    How many monitors do you use while coding?

    Results (600 votes). Check out past polls.