Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

RFC: IPC::System::Simple under Win32

by pjf (Curate)
on Jul 09, 2007 at 05:57 UTC ( #625565=perlmeditation: print w/ replies, xml ) Need Help??

Wise monks,

I'm working on a number of improvements to IPC::System::Simple, a cross-platform module designed to take the headache out of calling system() and checking its returns. Unfortunately, I've hit a snag when it comes to the behaviour of system() under Windows.

The idea of IPC::System::Simple is that you can replace this (from perldoc -f system):

system("some_command"); if ($? == -1) { die "failed to execute: $!\n"; } elsif ($? & 127) { die "child died with signal %d, %s coredump\n", ($? & 127), ($? & 128) ? 'with' : 'without'; } else { die "child exited with value %d\n", $? >> 8; }

with this:

use IPC::System::Simple qw(run); run("some_command");

Currently this works very well, but it's currently got a small problem detecting errors on Win32 systems.

Put very simply, any call to system() under Windows will fall-back to using the shell if it can't execute the command directly. This occurs regardless of the number of arguments, or the presence or absence of shell metacharacters. The problem with using the shell is that on failure to start the given command it returns an exit value of 1, which is indistinguishable from a command successfully running and merely returning an exit value of 1.

The end result is that we can never really be sure if Window's system() has failed or not.

Interestingly enough, the exact same problem occurs under Unix, but Perl only uses the shell under Unix when it's called with a single argument, and then only when it contains metacharacters. When the Unix shell is used, we can't tell the difference between a command failing to start, and one that runs to completion and merely returns an exit value of 127.

I'm intending for IPC::System::Simple to always mimic the Unix behaviour of system, regardless of the platform used. In other words:

use IPC::System::Simple qw(run); run("foo"); # No meta-chars, shell never used run("bar | bar"); # Meta-chars, shell always used. run("foo", "bar", "<baz") # Multi-arg, shell never used.

In the above examples, the first and third examples would be passed to the shell under Win32 if foo was not in the current working directory and we used system() instead. Neither would use the shell under Unix and modern releases of Perl.

One advantage of bypassing the shell is that we can also return the full 16-bit exit code returned by Windows processes. When using system() this is truncated to 8 bits.

This RFC is intended primarily as a sanity check. I feel I have a compelling reason to deviate from Perl's behaviour of system() under Windows; the current implementation is inconsistent compared to its Unix counterpart, it fails in an ambiguous way, and loses bits of the return value. Can anyone find me a compelling argument not to do this?

Further references:

Comment on RFC: IPC::System::Simple under Win32
Select or Download Code
Re: RFC: IPC::System::Simple under Win32
by xdg (Monsignor) on Jul 11, 2007 at 05:11 UTC
    Can anyone find me a compelling argument not to do this

    Tongue-in-cheek answers: Because having another module to do "system" doesn't actually make things simpler? Because having yet another special case of how things work on Windows doesn't make things simpler?

    In seriousness, I'm all for better portability and a friendlier API than "system", but this has been attempted so many times that I'd really rather see an attempt to bring together the best insights or try to merge new insights into one of the stronger existing modules than see an attempt to create yet another "special case". (See module list that follows.)

    Personally, I tend to favor IPC::Run3, but only because I've used it the most and had few problems with it across unix and Win32.

    Moreover, attempting to emulate Unix "system" behavior on Win32 strikes me as potentially quite tricky given all the special cases that had to be built into "system" on Win32 in the first place.

    For example, you suggest that a multi-arg list never be passed to the shell. So how would you replace/handle this call to "system":

    system("echo", "%PATH%"); # works fine on Win32

    On Win32, echo is a built-in command and has to be passed to cmd.exe and "%PATH%" does environment variable expansion (at some point anyway -- I think on the command line, but I'm not entirely sure. That works with "system" but I don't think it necessarily would with your "run" in IPC::System::Simple.

    Anyway, as you develop these ideas further, I encourage you to post them on http://win32.perl.org so that all may benefit.

    c.f other docs and modules (including ones I know and others that I just found searching on CPAN):

    -xdg

    Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

      Tongue-in-cheek answers: Because having another module to do "system" doesn't actually make things simpler?

      There's no denying that IPC::System::Simple is yet another wheel. It overlaps with a lot of the existing modules that handle calls to external commands. However it exists specifically to present the smallest possible learning curve for someone who wants to Do The Right Thing when calling external commands, but doesn't care how.

      I can take someone who has almost no understanding of how Perl works, who has a full bladder, and a flight to catch in five minutes, and still teach them how to use IPC::System::Simple. I can do it in one slide in a classroom.

      When I can tell people they can take their ugly, legacy Perl code, and replace each instance of system() with run() and suddenly get error checking and detailed diagnostics for free, that's a powerful force. If they want anything more than that, that's what all the other modules are for.

      Because having yet another special case of how things work on Windows doesn't make things simpler?

      I agree entirely! This is specifically making the module work the same way under Windows as it does everywhere else. Let's take your example:

      For example, you suggest that a multi-arg list never be passed to the shell. So how would you replace/handle this call to "system":

      system("echo", "%PATH%"); # works fine on Win32

      And here's the crux of the problem. Currently, that does work fine under Windows. According to the documention in perldoc -f exec, it shouldn't:

      Using an indirect object with "exec" or "system" is also more secure. This usage (which also works fine with system()) forces interpretation of the arguments as a multivalued list, even if the list had just one argu- ment. That way you're safe from the shell expanding wildcards or splitting up words with whitespace in them.

      @args = ( "echo surprise" ); exec @args; # subject to shell escapes # if @args == 1 exec { $args[0] } @args; # safe even with one-arg list

      Under Windows, Perl's system() is currently doing exactly what it shouldn't do, interpreting shell metacharacters and commands when called with multiple arguments. In my opinion, this is a bug in Perl.

      So, I guess my real question should be this. At what point does IPC::System::Simple break bug-for-bug compatibily with Perl, and do what Perl says it should do, rather than what Perl actually does?

        According to the documention in perldoc -f exec, it shouldn't:

        Perl documentation is Unix centric. It would be nice if there was at least a note in perlfunc for all functions that are cited in perlport. From the latter, regarding system:

        As an optimization, may not call the command shell specified in $ENV{PERL5SHELL}. "system(1, @args)" spawns an external pro‐ cess and immediately returns its process designator, without waiting for it to terminate. Return value may be used subse‐ quently in "wait" or "waitpid". Failure to spawn() a subprocess is indicated by setting $? to "255 << 8". $? is set in a way compatible with Unix (i.e. the exitstatus of the sub‐ process is obtained by "$? >> 8", as described in the documen‐ tation). (Win32)

        Of course, as you've demonstrated, this is incomplete as well, since apparently Perl may or may not call the shell based on some complex, undocumented logic.

        Worth noting here that you should consider what IPC::System::Simple does in the case of run(1, @args).

        I can take someone who has almost no understanding of how Perl works, who has a full bladder, and a flight to catch in five minutes, and still teach them how to use IPC::System::Simple. I can do it in one slide in a classroom.

        While teaching is a wonderful thing, I don't think modules should necessarily be designed to optimize teaching. In this case, I think the simplicity obscures critical understanding of just how tricky portable system calls are. Moreover, it leaves them with nothing to fall back on when their needs develop beyond IPC::System::Simple.

        As I look at the code of IPC::System::Simple, it seems to me that it does two primary things:

        • Encapsulates substantial logic for extracting return codes and exit signals from $? as portably as possible

        • Throws exceptions on errors or, optionally, certain return codes.

        I think the first would be ideally extracted into a module of its own so that it could be used with other alternatives for system as well.

        The second is a coding style preference.

        I personally don't like the "prepend an array of acceptable return values" feature -- I think that's another special case that makes things less simple. It also makes it really challenging to read code unless one already knows what IPC::System::Simple does:

        use IPC::System::Simple qw(run); my $exit_value1 = run("foo", @args); my $exit_value2 = run([0..5], "foo", @args);

        The first is pretty intuitive. The second is not. Again, from a teaching perspective, I'm not sure that's the right approach.

        Imagine an IPC::System::ExitValue module instead:

        use IPC::System::ExitValue qw/exit_value exit_signal/; system( "foo", @args ); croak "foo stopped early" if exit_signal($?); my $exit_value = exit_value($?);

        That would be trivial to write with the guts of IPC::System::Simple. And it could be used with other modules that substitute for system (as long as they preserve $?).

        If you really want to help throw exceptions for anything other than some exit values:

        use IPC::System::ExitValue qw/assert_exit/; system( "foo", @args ) and assert_exit( 0 .. 5 );

        I think that kind of approach would keep things simple for teaching but provide much greater functionality as students' and other Perl programmers' needs progress.

        -xdg

        Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://625565]
Approved by bingos
Front-paged by bingos
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (14)
As of 2014-10-30 17:43 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    For retirement, I am banking on:










    Results (208 votes), past polls