Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris

Re: Having to manually escape quote character in args to "system"?

by haukex (Monsignor)
on Sep 12, 2017 at 12:38 UTC ( #1199195=note: print w/replies, xml ) Need Help??

in reply to Having to manually escape quote character in args to "system"?

AFAIK it isn't as easy for Perl to avoid the shell on Windows, and the system docs do say "On Windows, only the system PROGRAM LIST syntax will reliably avoid using the shell; system LIST, even with more than one element, will fall back to the shell if the first spawn fails." So you might want to try system {$^X} $^X, ...;. Otherwise, there is Win32::ShellQuote, and I wrote an article about various other methods for running system commands, some of which also help you avoid the shell, here. In particular, newer versions of IPC::Run3 will automatically use Win32::ShellQuote on Windows.

Replies are listed 'Best First'.
Re^2: Having to manually escape quote character in args to "system"?
by salva (Abbot) on Sep 12, 2017 at 14:05 UTC
    AFAIK it isn't as easy for Perl to avoid the shell on Windows,

    The issue is not the shell.

    A windows program is not called with an array of arguments (as in Unix) but with a single command line and it is the program (not the shell) the one that breaks the command line into an array of arguments.

    To make things worse, every program may use its own rules to process the command line. Nowadays things are more or less standardized (see CommandLineToArgvW, introduced with Windows 2000 and Windows XP), but historically, every language supporting library would use its own variation (for instance, see C++), so in order to quote a command properly, you should take into account the program implementation language!

    In summary, the real issue is that in order to call a program in Windows with a list of arguments you have to quote and combine those arguments into a single command line and that quoting in Windows can be really tricky.

    Update: An interesting read: Everyone quotes command line arguments the wrong way.

      Everyone quotes command line arguments the wrong way is quite funny, in a sad way, and it is wrong. As wrong as any other program attempting to quote on Windows. It is a game that you simply can not win.

      You explained the basic problem: Arguments are passed to programs as a single string on systems derived from CP/M (i.e. DOS, Windows, OS/2), and programs (or the underlying runtime libraries) decide how to split that single string into arguments (see also Re^3: Perl Rename). Backwards compatibility to ancient DOS and WinNT, including bugs in and cmd.exe, have lead to a ridiculous amount of complex rules for quoting and escaping.

      The CommandLineToArgV convention mentioned in "Everyone quotes command line arguments the wrong way" is just that - a convention. All programs are free to use different quoting rules, and at least legacy programs do have different rules. (I did not look up or test, but I would not be surprised if cygwin-based programs would implement very different quoting rules, or even use a cygwin-only way to pass argv[] around, with a command line string only as fallback for non-cygwin programs.)

      Pretending that this convention is universal for all programs, and claiming that code that escapes and quotes according to the convention is the only correct solution, would be really funny, if it was posted by a noob in some dusty corner of the internet or our local universal expert. Posting that at is just sad.

      Unix has gone a long way, but the authors got argument passing right at the first attempt (i.e. fork() and exec()). And based on that lucky API, they made argument-splitting a problem of the shell, so you can use exactly the same quoting for all invoked programs. Over time, the shells got rid of most argument-splitting and argument-passing problems. That made quoting rules on Unix quite simple (but still far from being perfect). The best thing is that on Unix, you don't have to invoke the shell at all, so you don't have to quote at all. You pass a list of arguments to exec(), and main() will get exactly that list in argv[].


      Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

      Thank you very much for the details! I hardly ever run external commands on Windows, so I haven't gotten into the details very much, other than that Win32 apparently doesn't have an equivalent of execvp(3), and that Perl actually does its own quoting internally, which unfortunately doesn't seem to be perfect. But I have heard a few good things about Win32::ShellQuote, and I haven't had any problems with IPC::Run3 on Windows (although I may just have not yet run into a case of really complicated quoting).

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1199195]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (6)
As of 2017-10-20 09:38 GMT
Find Nodes?
    Voting Booth?
    My fridge is mostly full of:

    Results (260 votes). Check out past polls.