Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses

Re^4: Improve pipe open? (redirect hook)

by afoken (Chancellor)
on Apr 03, 2017 at 17:22 UTC ( #1186854=note: print w/replies, xml ) Need Help??

in reply to Re^3: Improve pipe open? (redirect hook)
in thread Improve pipe open?

Anyway, I was contemplating the numerous problems with piping/capturing I've witnessed on PM and elsewhere. Can you give an example where the list form open has caused mayhem, because of the one-element list?

You can run into trouble everywhere perl runs into something like exec @list or system @list, where @list may contain only one element. You currently have to write exec { $list[0] } @list or system { $list[0] } @list. If you don't, and assume that system or exec will fail if the program in $list[0] does not exist, you may have a security problem. @list=('rm -rf /') is no problem with system { $list[0] } @list (it will fail with a "file not found" error), but will cause a lot of trouble with system @list, because perl will invoke rm. It's a trap, but it is documented and should be known.

With open my $handle,'-|',@list, you will always run into that trap, because the indirect object ({ $list[0] }) that disables all code leading to the default shell can't be used with open. That's why I propose to add a flag to open so that there is a different way to disable code leading to the default shell.

As far as qx{}; is concerned,

qx/`` is not the point. qx is generally unportable and depends on the OS version due to the default shell behavior, with a few exceptions where all default shells behave the same or perl does not invoke the default shell.

Unsafe pipe open (with up to three arguments for open) has the same problem. Safe pipe opens from perlipc with exec { $list[0] } @list in the child process completely avoids the default shell. Pipe open with at least four arguments for open (as implemented since perl 5.8.0) also avoids the default shell.

I do not really see any problem. The string inside qx is not perl code, it is shell syntax. One could perhaps make a point about always requesting a shell, even when perl thinks this is redundant, like qx{}F; maybe. The opposite, to force an op to not do what it's intended to do, makes no sense.

The string inside qx should be what you call "shell syntax", yes. So please define "shell syntax". Start with quoting rules that work for all shells. Have a look at and to get a feeling for the fun you will have.

Let me list some default shells:

  • bourne shell (sh)
    • System III
    • SVR2
    • SVR3
    • SVR4
    • BSD
  • C shell (csh)
    • csh
    • tcsh
  • Korn shell (ksh)
    • pdksh
    • ksh93
    • mksh on Android
  • Bourne Again Shell (bash)
    • v1
    • v2
    • v3
    • v4
  • Almquist Shell (ash)
    • Original
    • Debian's fork of Almquist Shell (dash)
    • NetBSD fork
    • Busybox's fork
  • zsh
  • Plan9 shell (rc)
  • Windows'
  • Windows' cmd.exe
  • on OS/2

All of these shells come in different versions, and they all have different behaviour when parsing strings into commands and arguments. See for just a few of the many problems with shell behaviour on unixoid systems. And gues what happens when you feed a string intended for some unix shell to or vice versa.

That's the first problem with "the" shell. There is no single shell on every operating system that behaves the same on every operating system. The default shell is not even consistent across different versions of the same OS. See for a quite long list of default shells.

The second problem is that perl guesses what may happen when "the" shell parses the string and sometimes tries to avoid "the" shell. Have a look at Perl_do_exec3() (see Re^2: Improve pipe open? (redirect hook)) to see details. And no, it's not only the misterious "shell metacharacters" that trigger using the shell.

Action-at-a-distance was precisely the intention in this case. One might then trivially enhance a standard capture with a certain additional effect like dropping of privileges. Callbacks like that allow for a (more) generalized routine instead of a bunch of specialized modules.

Action-at-a-distance is an excellent way to create unmaintainable, write-only code.

Imagine a small, but not tiny, old project that makes use of qx. Code is spread over several modules, and it runs fine on current perl. I would say this is a quite common scenario. Now imagine that project needs a new feature. A coworker wraps it in a new module like this and commits to CVS, SVN, git or whatever.

package New::Module; use strict; use warings; # 250 lines later: sub prepare_foo { # ... $SIG{'__EXEC__'}='special_foo'; # ... } # 100 lines later: sub do_foo { # ... my @text=`foo \$BAR` # ... } # 200 lines later: sub finish_foo { # ... $SIG{'__EXEC__'}=''; # ... } # 80 lines later: sub special_foo { # ... open STDOUT,">&STDERR"; # ... } # and 500 more lines 1;

Now, customers report tons of bugs. Whle debugging, you find out that every single qx/`` in every single module suddenly behaves mad. Guess why.

Edit. BTW: out of curiosity, do you sometimes use the <> operator in your code or do you always go for the safe diamond? I'd have simply plugged *that* hole, methinks...

I don't use either, since years. I found a single old and unused script in a dark corner of my all-knowing, all-seeing SVN repository that uses <>.


Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1186854]
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others having an uproarious good time at the Monastery: (2)
As of 2023-12-11 03:18 GMT
Find Nodes?
    Voting Booth?
    What's your preferred 'use VERSION' for new CPAN modules in 2023?

    Results (41 votes). Check out past polls.