Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

capturing command output

by morgon (Deacon)
on Jan 25, 2012 at 10:03 UTC ( #949863=perlquestion: print w/ replies, xml ) Need Help??
morgon has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

in some legacy code (running on Windows) we came across this to capture the output of an external command:

sub fetch_cmd { my ($cmd) = @_; my $ret; unless ( open FH, $cmd . ' 2>&1 |' ) { return; } while (<FH>) { $ret .= $_; } close FH; return $ret; }
We are currently having some discussions on whether the above construct involves a shell (I think it does) and whether it is better then simply using backticks (I think it is not).

What is your opinion?

Comment on capturing command output
Download Code
Re: capturing command output
by BrowserUk (Pope) on Jan 25, 2012 at 10:20 UTC

    It definitely does involve a shell; and I see no advantage, and several disadvantages over backticks.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    The start of some sanity?

Re: capturing command output
by Ratazong (Prior) on Jan 25, 2012 at 10:21 UTC

    Two aspects you might want to consider

    • writing to a temporary output file (as done in your example) might lead to troubles with write-access-rights
    • writing to a temporary output file is helpful when debugging, as that file can be checked easily afterwards outside of perl (unless you delete it)
    In my projects, I have no real preference. I typically decide it case-base-case based on the "feeled risk of needing debugging info"

    HTH, Rata

    Update: Oooops! I have overlooked the pipe-symbol. So please ignore my answer above! Thanks for pointing this out, BrowserUk!

      The code is not writing to a temporary file. The external process is writing directly to a pipe which is then read in the while loop.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

      The start of some sanity?

Re: capturing command output
by moritz (Cardinal) on Jan 25, 2012 at 10:35 UTC

    I'm pretty sure it does involve a shell; what other part should take care of the redirection 2>&1?

    And I don't think it's any better than backticks. It also lacks proper error handling.

      what other part should take care of the redirection 2>&1?

      Perl itself?  Like it does on Unix when 2>&1 is the only shell meta characters in the command.*

      I don't think Perl is making this optimisation on Windows, though, but I do think it's a valid question — considering that in Perl things are not always quite as obvious as it might seem at first...

      ___

      * as you can easily verify:

      $ strace -fqeexecve perl -e 'open FH, "echo foo bar 2>&1 |"; print <FH +>' execve("/usr/bin/perl", ["perl", "-e", "open FH, \"echo foo bar 2>&1 | +\"; "...], [/* 77 vars */]) = 0 [pid 1819] execve("/bin/echo", ["echo", "foo", "bar"], [/* 77 vars */ +]) = 0 foo bar

      No shell involved here.  But as soon as you add another shell metacharacter, e.g. ";", Perl will use the shell:

      $ strace -fqeexecve perl -e 'open FH, "echo foo bar ; 2>&1 |"; print < +FH>' execve("/usr/bin/perl", ["perl", "-e", "open FH, \"echo foo bar ; 2>&1 + |\""...], [/* 77 vars */]) = 0 [pid 1833] execve("/bin/sh", ["sh", "-c", "echo foo bar ; 2>&1"], [/* + 77 vars */]) = 0 foo bar

        Very interesting. ++

        I had no idea that perl did that optimization, it is good to know, but it would be nice to have a way to control it.

Re: capturing command output
by JavaFan (Canon) on Jan 25, 2012 at 11:17 UTC
    Considering all you do is run a command, and collect its output and errors, and return a string with said output and errors, I see no advantage of not using backticks. The backticks will also use the shell, but that doesn't really matter. Your fetch_cmd basically is a reimplementation of backticks.

      The OP's script captures both STDOUT and STDERR, but plain backticks will only capture STDOUT.

      As a demo, I used a script like this:

      #!/usr/bin/perl print STDOUT "hello\n"; print STDERR "world\n";

      If you run it on a bash cmd line you get both lines of output as you would expect. If you call via perl backticks then only "hello" is captured, and "world" is still sent to stdout.

        If you call via perl backticks then only "hello" is captured, and "world" is still sent to stdout.
        I cannot reproduce that.
        $ cat foo #!/usr/bin/perl print STDOUT "hello\n"; print STDERR "world\n"; $ cat bar #!/usr/bin/perl use 5.010; my $output = `./foo 2>&1`; say "[[$output]]"; $ chmod +x foo $ chmod +x bar $ ./bar [[world hello ]] $
        Or are you suggesting the OP would run a different command in backticks than in his open? Why would he do that?
      > Your fetch_cmd basically is a reimplementation of backticks.
      Yes. But besides handling stderr it gives you another layer of indirection. And as the saying goes: there is not problem you cannot address with another layer of indirection ;-) Here it might actually be useful for logging the call (if it's generated) or storing the output for later inspection (it may not look as expected) or discard stderr or...
        But besides handling stderr it gives you another layer of indirection.
        So what? You have any idea how many layers of indirection there are on the open call? Or on the reading? Why is the number of layers of redirection an argument against backticks, but not about open, or the diamond operator?
        Here it might actually be useful for logging the call (if it's generated) or storing the output for later inspection (it may not look as expected) or discard stderr or...
        Sure, but he isn't doing any of that. One can critic any piece of code with "yeah, but if you want to do X in the future, it becomes harder". With arguments like that, you'd never write down a single line of code.
Re: capturing command output
by eyepopslikeamosquito (Canon) on Jan 26, 2012 at 08:32 UTC

    We tend to avoid the Windows shell in Perl scripts that run external commands in customer environments over which we have no control. Some years ago we got burnt because some of our customers use a misfeature of Windows cmd.exe, namely that without the /d option, cmd.exe executes stuff from one of the following Registry values:

    HKEY_LOCAL_MACHINE\Software\Microsoft\Command Processor\AutoRun HKEY_CURRENT_USER\Software\Microsoft\Command Processor\AutoRun
    before running your command. In one case, I remember there was a "cd" command in the customer autorun Registry which caused one of our Perl scripts to malfunction. We worked around it by replacing `some command 2>&1` with `cmd /d/c "some command" 2>&1`. An alternative workaround is to manipulate STDOUT and STDERR from perl, while taking care to call the Perl system function in such a way that the shell is not invoked.

    To clarify, the two approaches are illustrated by the following two example programs:

    use strict; use warnings; my $cmd = 'perl -le "print qq{to stdout};print STDERR qq{to stde +rr};exit 42"'; my $cmdhack = qq{cmd /d/c $cmd 2>&1}; # works # my $cmdhack = qq{cmd /d/c "$cmd" 2>&1}; # also works (despite bizar +re quoting) print "run:$cmdhack:\n"; my $out = `$cmdhack`; my $rc = $? >> 8; print "rc=$rc, stdout/stderr of command follows:\n"; print $out;

    and:

    use strict; use warnings; my $outfile = 'ff.tmp'; my $exe = $^X; my @args = ( 'perl', '-le', 'print qq{to stdout};print STDERR qq{to + stderr};exit 42' ); print "run exe:$exe:args:@args:\n"; print STDERR "here we go to stderr\n"; open(SAVOUT, ">&STDOUT") or die "error: save original STDOUT: $!"; open(SAVERR, ">&STDERR") or die "error: save original STDERR: $!"; open(STDOUT, '>', $outfile) or die "error: create '$outfile': $!"; open(STDERR, '>&STDOUT') or die "error: redirect '$outfile': $!"; system { $exe } @args; my $rc = $? >> 8; open(STDOUT, ">&SAVOUT") or die "error: restore STDOUT: $!"; open(STDERR, ">&SAVERR") or die "error: restore STDERR: $!"; close(SAVERR) or die "error: close SAVERR: $!"; close(SAVOUT) or die "error: close SAVOUT: $!"; print "rc=$rc\n"; print STDERR "rc=$rc to STDERR\n";
    Improvements welcome.

    Update: tidied up and clarified the example code.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://949863]
Approved by Corion
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (7)
As of 2014-07-26 07:41 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (175 votes), past polls