Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

sed/awk/grep to Perl

by aartist (Pilgrim)
on Nov 09, 2020 at 14:57 UTC ( #11123527=perlquestion: print w/replies, xml ) Need Help??

aartist has asked for the wisdom of the Perl Monks concerning the following question:

Inherited few Perl Script with echo/sed/awk/grep commands in backticks or system. Is it best practice to convert them to Perl or to leave as it is. Do we have conversion recopies ? I do not need Perl one-liners to replace with.

Replies are listed 'Best First'.
Re: sed/awk/grep to Perl
by choroba (Archbishop) on Nov 09, 2020 at 15:15 UTC
    If they work and you have other urgent tasks, it's OK to leave them as they are.

    The main benefits of not shelling out are a performance and safety improvements: A single Perl process usually works much faster than Perl calling a shell calling grep etc. And as variables are real variables in Perl (while in the shell, we only have simple expanding macros), you don't need to worry about whitespace or quotes in their values.

    map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]
Re: sed/awk/grep to Perl
by haukex (Bishop) on Nov 09, 2020 at 15:15 UTC
    Inherited few Perl Script with echo/sed/awk/grep commands in backticks or system. Is it best practice to convert them to Perl or to leave as it is.

    IMHO, yes, definitely! See Calling External Commands More Safely for the issues with backticks, and also The problem of "the" default shell, which also names a few equivalencies.

    Do we have conversion recopies ?

    Well, for sed and awk there are App::s2p and App::a2p, respectively, although note the Perl code they produce isn't perfect - but it's a good starting point for conversion. As for the others, grep shouldn't be all too difficult to convert to Perl regexes, and if there's anything else, then perhaps you can name them more specifically and show examples, and perhaps we can help with those.

Re: sed/awk/grep to Perl
by BillKSmith (Prior) on Nov 09, 2020 at 19:18 UTC
    You seem to be getting a divided opinion. Let me suggest a middle ground. Do not make the suggested changes until you must change the script for any other reason. Then yes, make all the improvements. The benefit of merely changing to pure perl, although real, is probably not worth the effort of validating the new version. If you wait until requirements change, you will already need the validation.
    Bill
      You seem to be getting a divided opinion. Let me suggest a middle ground. Do not make the suggested changes until you must change the script for any other reason. Then yes, make all the improvements. The benefit of merely changing to pure perl, although real, is probably not worth the effort of validating the new version. If you wait until requirements change, you will already need the validation.

      I'm not really sure that's a middle ground, rather it seems to be quite similar to choroba's point. I think the "real answer" is: it depends. Since aartist hasn't shown any real-world examples, we don't know if the code in the backticks is innocuous, or whether it depends on a specific shell (brittle) or puts user input into backticks - a major potential security issue. If I were to receive, for example, some legacy C code and I discovered a buffer overflow, I'd want to fix it sooner rather than later, but of course it could be that there are other mitigating factors, like being able to assume the input this program receives is sanitized. In other words, it should be decided on a case-by-case basis what the priority to change the code should be. As a general response, like in my post, I erred on the side of caution, because in my experience backticks without variables interpolated into them are rare.

Re: sed/awk/grep to Perl
by eyepopslikeamosquito (Bishop) on Nov 09, 2020 at 22:04 UTC

    Inherited few Perl Script with echo/sed/awk/grep commands in backticks or system. Is it best practice to convert them to Perl or to leave as it is
    Unix shell versus Perl argues that it's better to write everything in Perl to begin with (rather than a motley mix of perl/sed/awk/grep).

    Re when/if to rewrite working legacy code, I keep a long list of previous PM discussions on that tricky topic at: Re: Strategies for maintenance of horrible code?

Re: sed/awk/grep to Perl
by stevieb (Canon) on Nov 09, 2020 at 17:08 UTC

    As the other experienced Monks have said, yes, it's definitely worth it.

    However, you should write a comprehensive unit test suite against the code you currently have before changing anything, so that the new code can also be tested against those tests to ensure that it operates at minimum exactly the same way. You should also ensure your code is in a Version Control System of some sort, so that you can easily revert out changes that break things.

      However, you should write a comprehensive unit test suite against the code you currently have before changing anything, so that the new code can also be tested against those tests to ensure that it operates at minimum exactly the same way.

      I just wanted to note that it's also possible to write the tests just against the command being run - luckily the interface to external commands is well-defined (usually just the commandline, STDOUT, and perhaps some files), so it's not too difficult to write a comprehensive test suite that tests the commands in the system or backticks, and then run that test suite against its pure-Perl replacement. Although I agree that a comprehensive test suite against the entire program would of course be very helpful and should eventually be built, the aforementioned approach allows one to make changes and write tests for a smaller scope first.

Re: sed/awk/grep to Perl
by jszinger (Scribe) on Nov 09, 2020 at 22:13 UTC

    In my experience the use of backticks or system to do what Perl can do indicates a lack of quality—there are often problems lurking in the rest of the code since the original programmer didn’t know Perl very well. Here be dragons. If you look at it funny, it might break.

    I stopped writng awk scripts the day I learned Perl. There’s nothing that awk can do that cannot be easily done in Perl. I find grep handy from the command line and do simple sed calls from shell scripts and Makefiles. I am never tempted, however, to call them from Perl.

    For added fun, consider that different implementations of awk, grep, and sed have incompatible feature sets and regular expression syntax.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://11123527]
Approved by haukex
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (4)
As of 2021-05-18 14:02 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Perl 7 will be out ...





    Results (180 votes). Check out past polls.

    Notices?