Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine

At what point do you rewrite that old shell script in Perl?

by rudder (Scribe)
on Apr 11, 2008 at 20:24 UTC ( #679844=perlquestion: print w/replies, xml ) Need Help??
rudder has asked for the wisdom of the Perl Monks concerning the following question:

You've got a number of shell and Perl scripts under your care. Lately you've been eyeing a certain overgrown sometimes-problematic-but-generally-working shell script with a hankerin' to rewrite it in Perl -- including adding a few wishlist features to it while you're at it.

At what point do you rewrite that old shell script in Perl? When it's 100 lines long? 500? 1000?

  • Comment on At what point do you rewrite that old shell script in Perl?

Replies are listed 'Best First'.
Re: At what point do you rewrite that old shell script in Perl?
by kyle (Abbot) on Apr 11, 2008 at 20:41 UTC
Re: At what point do you rewrite that old shell script in Perl?
by perlfan (Curate) on Apr 11, 2008 at 20:47 UTC
    When you have to start munging data or can't readily rely on commandline utilities to do what you need but stringing them together with pipes, it is time to switch from shell to Perl.

    In general, a shell program should not be that large or take too many commandline options. For that matter, Perl utilities shouldn't either. No script, no matter what language, should be too big - nor should it be a kitchen sink type app that tries to do too much. Remember, do one thing, and do it well :)

    I suggest trying to keep to small shell scripts that bind together specialized utilities. You may find it nicer to create the specialized utilities in Perl, then use them inside of a shell script. The unix "everything is a filter" mantra works pretty well, too, if you separate things properly.

Re: At what point do you rewrite that old shell script in Perl?
by runrig (Abbot) on Apr 11, 2008 at 21:19 UTC

    It's not so much the length, it's whether or not perl data structures, regular expressions, modules, etc., would make the program "better". I still write Korn shell scripts, and one I'm working on now is 140 lines long (it's mainly a wrapper for some external commands), and there's currently no reason to write it in perl, though it wouldn't be too hard to do if it became necessary.

    If you do program in Korn shell, I recommend learning about its (regex-like) patterns, parameter expansions, command substitutions, compound commands, background processes, and co-processes (and whatever of this that applies if it's bash or some other shell). These things can push back the need for rewriting in perl.

Re: At what point do you rewrite that old shell script in Perl?
by mr_mischief (Monsignor) on Apr 11, 2008 at 20:54 UTC
    If pressed for a number of lines, I'd say about 25, give or take 15 lines. Perl offers far more structure and error handling than pretty much any everyday shell. Perl also tends to be more legible if written in common styles and using common idioms than shell does.

    The real reasons I'd rewrite aren't quite as straightforward as a number of lines, though. If your shell script is just a driver for a bunch of other programs which run pretty much in order with fairly static sets of command-line options, then it can be fairly long without being an issue at all. If, however, it is less of a scriptable harness and more of an application in its own right, then it probably should have been written in a proper general purpose programming language in the first place.

      If your shell script is just a driver for a bunch of other programs which run pretty much in order with fairly static sets of command-line options, then it can be fairly long without being an issue at all.

      Right; many scripts start life exactly like that. For example, "check out some files, run make, check condition X and scp files to server Y, update config file Z...". Many lines of the script are simply using external programs like cvs, scp, mysqldump, whatever. If it's just a page full of lines like that, I wouldn't want a long sequence of system calls in a Perl script.

      But as time goes by, users want more and more little bits added here and there, and even though it's still primarily a "driver for a bunch of other programs", now it's starting to do a little more error checking, more branching, and repeated bits are put into functions.

      This is primarily the case I'm asking about here. When it's a sort of gray area, and you know you could just dig in and update the shell script, but you also know that it really won't fix the more inherent problems that a rewrite in Perl would. Also you want less work for yourself the next time you get a feature request for said script.

        Here are a few rules of thumb I use. Your thumbs may be of a different size than mine (which would be quite probable if I was speaking literally).

        I'd say any shell script that is subject to being updated as the result of a feature request is a strong candidate for rewriting in a more powerful language. Most shell scripts are part of an ad hoc implementation, a prototype of something, are fairly simple, are needed because more powerful languages aren't available (like at boot time for init scripts), or are used specifically because they can alter the environment of the login shell. If you're having to maintain changes to a shell script that's not bound to stay a shell script by circumstance then it's a good time to rewrite it.

        Any time updating a program and rewriting (or refactoring) the program to be more maintainable are in the same ballpark in terms of time, opt for making it more maintainable later. This goes double if you're rewriting in a more suitable language precisely because the old language doesn't support the changes well. This goes for all languages and not just shell.

        Any time a shell script is calling something written in Perl, take the time to figure out why you're calling that functionality from the command line instead of as a sub. If you have a good answer, consider not rewriting the shell script. If you don't, it's probably a good time to plan a rewrite.

        If the startup times of your independent programs called by the shell script is slowing the script down too much (for some value of "too much") and you could easily duplicate most of the functionality of those utility programs in a few minutes of coding Perl, then go ahead and do most of the work in Perl and only shell out to the parts that are harder to duplicate.

        If you foresee the need to run in disparate environments (like Windows and Linux) where you can choose fighting with some implementation of your shell for a nonstandard platform or just installing Perl (or using an existing Perl installation on the new target if it's there), then rewrite in Perl.

        If you're having to write comments about program flow because your shell script is getting difficult to follow, then you need it to be in another, more powerful language.

        If you're planning to handle more data than the command-line tools you're calling from the shell script handle well, Perl is an excellent solution.

        Any situation in which your environment variables are suspect and the command-line tools you're calling change their behavior based on environment variables, you might want Perl just for the sake of things like easily checking/altering the whole $ENV hash and Shell scripts that do CGI have always, for example, made me nervous.

        Anything repetitive that is difficult to modularize and reuse is a sign you need a more flexible language. This is especially the case when it's because your language lacks a way to easily handle the type of conditional execution you need for the parts that aren't exactly the same.

        This list isn't exhaustive, but it just might be exhausting. ;-) In any case, the best rule of thumb is probably that if you have to stop to consider rewriting a shell script in Perl, that you're going to be better off doing the rewrite.

Re: At what point do you rewrite that old shell script in Perl?
by dragonchild (Archbishop) on Apr 11, 2008 at 23:17 UTC
    Rewriting anything that currently works is dangerous and should only be undertaken when the product in question cannot be extended to a specific enhancement need without an inordinate amount of unsafe practices. This goes for any piece of code, period.

    My criteria for good software:
    1. Does it work?
    2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?

      I'll add -- or you have to port it to another environment, and it has dependancies that you can't rely on.

      Eg, if I'm going to have to move from a BSD to an ATT unix, and do something with parsing ps output, and suddenly have to deal with the difference in options between systems. (it's one thing to move it completely, but I don't want to maintain two variants if I don't have to.

      Well, that said, things aren't so clear cut. Often, existing scripts limp along with tolerable inadequacies. And maybe they already use unsafe practices, but those transgressions have been overlooked since the scripts work most of the time, and when they don't, it's not too much work to clean up messes and run the script again.

      Thanks for your answer though. As I read it, the answer is: you've got to use your judgement and consider how hard it will be to add further enhancements and how unsafe the current script is.

Re: At what point do you rewrite that old shell script in Perl?
by FunkyMonk (Chancellor) on Apr 11, 2008 at 21:03 UTC
    As soon as I can, or sooner ;)

    If writing from scratch, as soon as I realise I do need an if/while/until... after all.

Re: At what point do you rewrite that old shell script in Perl?
by oko1 (Deacon) on Apr 12, 2008 at 04:19 UTC

    It's not really linecount-based, for me - I'll do it as soon as the need for calculation or complex processing becomes significant, or if it's apparent from the beginning. Contrariwise, if the task mostly requires filesystem operations rather than either of the above, I'll stick with a shell script. I've even been known to do a back-ticked Perl op in a shell script that's doing heavy filesystem lifting - and have also done Perl scripts with the occasional back-ticked shell op. Having both gives me a lot of flexibility/granularity along that spectrum.

    Human history becomes more and more a race between education and catastrophe. -- HG Wells
Re: At what point do you rewrite that old shell script in Perl?
by sundialsvc4 (Abbot) on Apr 11, 2008 at 20:50 UTC

    Given that you have Perl at your disposal ... why use shell scripts? At all?

      When you inherit the scripts, and then users ask for minor changes and it's easier to make those minor changes than dive in and rewrite the whole thing.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://679844]
Approved by FunkyMonk
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others having an uproarious good time at the Monastery: (2)
As of 2018-07-23 04:28 GMT
Find Nodes?
    Voting Booth?
    It has been suggested to rename Perl 6 in order to boost its marketing potential. Which name would you prefer?

    Results (459 votes). Check out past polls.