in reply to Re: Having to manually escape quote character in args to "system"?
in thread Having to manually escape quote character in args to "system"?

AFAIK it isn't as easy for Perl to avoid the shell on Windows,

The issue is not the shell.

A windows program is not called with an array of arguments (as in Unix) but with a single command line and it is the program (not the shell) the one that breaks the command line into an array of arguments.

To make things worse, every program may use its own rules to process the command line. Nowadays things are more or less standardized (see CommandLineToArgvW, introduced with Windows 2000 and Windows XP), but historically, every language supporting library would use its own variation (for instance, see C++), so in order to quote a command properly, you should take into account the program implementation language!

In summary, the real issue is that in order to call a program in Windows with a list of arguments you have to quote and combine those arguments into a single command line and that quoting in Windows can be really tricky.

Update: An interesting read: Everyone quotes command line arguments the wrong way.

  • Comment on Re^2: Having to manually escape quote character in args to "system"?

Replies are listed 'Best First'.
Re^3: Having to manually escape quote character in args to "system"?
by afoken (Canon) on Sep 12, 2017 at 22:00 UTC

    Everyone quotes command line arguments the wrong way is quite funny, in a sad way, and it is wrong. As wrong as any other program attempting to quote on Windows. It is a game that you simply can not win.

    You explained the basic problem: Arguments are passed to programs as a single string on systems derived from CP/M (i.e. DOS, Windows, OS/2), and programs (or the underlying runtime libraries) decide how to split that single string into arguments (see also Re^3: Perl Rename). Backwards compatibility to ancient DOS and WinNT, including bugs in and cmd.exe, have lead to a ridiculous amount of complex rules for quoting and escaping.

    The CommandLineToArgV convention mentioned in "Everyone quotes command line arguments the wrong way" is just that - a convention. All programs are free to use different quoting rules, and at least legacy programs do have different rules. (I did not look up or test, but I would not be surprised if cygwin-based programs would implement very different quoting rules, or even use a cygwin-only way to pass argv[] around, with a command line string only as fallback for non-cygwin programs.)

    Pretending that this convention is universal for all programs, and claiming that code that escapes and quotes according to the convention is the only correct solution, would be really funny, if it was posted by a noob in some dusty corner of the internet or our local universal expert. Posting that at is just sad.

    Unix has gone a long way, but the authors got argument passing right at the first attempt (i.e. fork() and exec()). And based on that lucky API, they made argument-splitting a problem of the shell, so you can use exactly the same quoting for all invoked programs. Over time, the shells got rid of most argument-splitting and argument-passing problems. That made quoting rules on Unix quite simple (but still far from being perfect). The best thing is that on Unix, you don't have to invoke the shell at all, so you don't have to quote at all. You pass a list of arguments to exec(), and main() will get exactly that list in argv[].


    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
Re^3: Having to manually escape quote character in args to "system"?
by haukex (Archbishop) on Sep 12, 2017 at 19:17 UTC

    Thank you very much for the details! I hardly ever run external commands on Windows, so I haven't gotten into the details very much, other than that Win32 apparently doesn't have an equivalent of execvp(3), and that Perl actually does its own quoting internally, which unfortunately doesn't seem to be perfect. But I have heard a few good things about Win32::ShellQuote, and I haven't had any problems with IPC::Run3 on Windows (although I may just have not yet run into a case of really complicated quoting).