opening '-|' on Windows

[Tanktalus]

In Automated module install, I started to look at creating a simple script to untar some tarballs, and automatically install them. I have something that, at least before I started mangling it, was working on Linux. So I tried on Windows. Didn't get too far, I might mention. Perhaps someone here with a bit more Windows experience than I could shed some light.

Environment: ActiveState 5.8.0, WindowsXP, and lots of Cygwin tools. (I'm hoping we get this upgraded to 5.8.6 soon - so if that's the solution, I'll just apply that pressure.)

The command works fine if I run it at the command prompt, so it's not a lack of the proper tooling (gunzip and tar are both present).

=item do_pipe_command

Mostly-simply wrapper around open for piping from a command.
Since we generally want to capture stderr, too, we redirect stderr
to stdout in the command, so we get it all in a single filehandle.

=cut

sub do_pipe_command
{
    print 'Running: ', join(' ', @_[1..$#_]), $/;

    my $pid = open ($_[0], '-|');
    return $pid if $pid;

    # child - pipe the stderr to the stdout before exec'ing.
    open STDERR, '>&', \*STDOUT;
    my @cmd = @_[1..$#_];
    if ($^O =~ /MSWin/i)
    {
        @cmd = (qw(cmd.exe /c), join(' ',@cmd));
    }
    exec @cmd;
}

=item extract_tarball

Tries to untar the tarball.  Returns the directory name thusly
created.

=cut

sub extract_tarball
{
    my $tarball = shift;

    my $cmd = '';
    if ($tarball =~ /\.tar\.gz$/)
    {
        $cmd = "gunzip -c $tarball | ";
    }
    elsif ($tarball =~ /\.tar\.Z$/)
    {
        $cmd = "zcat $tarball | ";
    }
    elsif ($tarball =~ /\.tar$/)
    {
        $cmd = "cat $tarball | ";
    }
    else
    {
        die "Can't figure out how to untar $tarball";
    }

    $cmd .= 'tar -xvf -';
    print "Running $cmd\n";

    my $subdir;
    do_pipe_command my $tar, $cmd or die "Failed to run $cmd: $!";
    while (my $l = <$tar>)
    {
        print $l;
        next if $subdir;
        $subdir = $1 if $l =~ m:^(.*?)[/\\]:;
    }
    $subdir;
}

The problem seems to be where I try to run the open. fork works fine that I can test, but this seems to go all wonky on me. The error message is:

Running gunzip -c Cache-Repository-0.01.tar.gz | tar -xvf -
Running: gunzip -c Cache-Repository-0.01.tar.gz | tar -xvf -
'-' is not recognized as an internal or external command,
operable program or batch file.
Use of uninitialized value in string eq at e:/perl5.8.0/lib/File/Spec/
+Unix.pm line
 77.

Yet, as I said, if I run the given command at the command prompt manually, it works fine.

I just installed ActiveState 5.8.6 ... different uninitialized value error message, but same "not recognized" message.

As I'm writing this, I'm starting to wonder if opening '-|' is even going to work on Windows, or am I going to have to resort to some more trickery to get what I want? Namely, a way to get the directory that the untar command is doing. (I wish Archive::Tar and Compress::Zlib were part of the standard perl distribution!)

Update: After reading tye's recommendation, I noticed IPC::Open3 has documented that this is a problem, so I'm going to have to replace the open call with something else. Thanks!

Update: As a confirmation, the new do_pipe_command is:

sub do_pipe_command
{
    print 'Running: ', join(' ', @_[1..$#_]), $/;
    my @cmd = @_[1..$#_];

    my $wtr;
    require IPC::Open3;
    my $pid = IPC::Open3::open3($wtr, $_[0], undef, @cmd);
}

It works perfectly. No need to launch cmd.exe (which may be because my @cmd is always a single string anyway, at least when I'm chaining subprocesses together in their own pipe). I do realise that this will leave a zombie process behind (at least on unix), but that should be automatically cleaned up when we exit, and this is a very short-lived process anyway. It runs on Linux in under 5 seconds currently, so a few zombies is pretty minor.

[jimbojones]

Isn't the problem summarized in perlipc "Safe Pipe Opens"

Note that these operations are full Unix forks, which means they may not be correctly implemented on alien systems. Additionally, these are not true multithreading. If you'd like to learn more about threading, see the modules file mentioned below in the SEE ALSO section.

I don't think you can fork on Windows
- j

[Tanktalus]

As stated above, I've found that fork is working (which was a surprise to me, too, until joining PM and being corrected on it myself ;->). But maybe -| isn't implemented.

[tye]

fork is emulated using threads but threads share file handles so you can't (without quite a bit more fakery being implemented) emulate pipes (between threads).

You should be building around something portable like IPC::Run or IPC::Open3 instead.

[BrowserUk]

Piped opens ("prog |" & "| prog") work, just not the '-|' varient. For some reason it has never been implemented.

[jimbojones]

Thanks for the updates. I'm learning all the time here ...
- j

[NetWallah]

This is just a wild guess, but I think you may need the "cmd" shell to process the "|" pipes.

Instead of starting your cmd with "tar ...", try
"cmd /c tar ...".

This will give the windows shell (cmd.exe) a chance to manage the pipe.

[Tanktalus]

Actually, I tried that. If you look closely at the do_pipe_command function above, I run (qw(cmd /c), join ' ', @cmd). However, I don't think I ever get that far - I think it's the opening of "-|" that causes the failure much earlier in the same sub. It's when I changed it to use cmd /c explicitly that I started to think there was something more sinister going on ;-)