Premature close of the pipe

[Superfox il Volpone]

Dear perl monks,
I am struggling to keep my pipe open long enough so that the underlying command has the time to complete.
This is my code:

  state $rsync_base_cmd = "rsync -avm -L --exclude='S/' --include='*/'
+ --include='*.cpp' " .
     "--include='*.hpp' --exclude='*'";
  
  # perform the copy
  my $command = "$rsync_base_cmd ${login}:${remoteDirectory} $localDir
+ectory";
  #print("\n$command\n"); # for debugging
  #`$command`; 
  my $output = ""; 
  my $pid = open3(undef, my $pipe, undef, $command) 
    or confess("[_copy_component] Cannot perform rsync : $!");
    
  while(<$pipe>){ $output .= $_; } # wait it completes to process
  
  eval { close ($pipe); };
  
  waitpid($pid, 0);

Unfortunately the pipe terminates before the command is able to copy all the involved files. But if I print the output into the stdout, then the pipe will correctly finish and all the files will be copied.

Can you please tell me how to ensure the underlying command completes without having to print the output into the stdout?

Kind regards,
Superfox

[zentara]

I think you need to use select on your pipe. Here is an example.

#!/usr/bin/perl -w
use strict;
use IPC::Open2;
use IO::Select;
$| = 1; # autoflush STDOUT

# Declare filehandles and command to use:
my ($r, $w);
my $cmd = 'ping 127.0.0.1';

# Open the process and set the selector:
my $pid = open2($r, $w, $cmd);
my $selector = IO::Select->new($r);

while (1)  # infinite loop (use "last" to break out)
{
if ($selector->can_read(0))
{
my $chars;
my $bytes_read = sysread($r, $chars, 4046);
print "$bytes_read $chars\n";
}

# Do anything you want in between reads here...
}

__END__

#by J. Romano

# The advantage to this script is that, if your commands (like
# "whois", "dig", and "ping") happen to pause, the loop won't
# automatically break out.  The disadvantage to this script is that it
# might be difficult figuring out when a command has finished, or just
# has delayed output (in which case you might have to put in a few
# sleep() calls).  Either way, I think that this script here does a
# better job of helping you visualize what is going on -- you just nee
+d
# to be mindful of the fact that some programs don't flush their outpu
+t
# right away, and that it's not a simple matter to tell if the program
# has stopped running altogether.

[McA]

Hi,

it is not a direct answer to your question and I don't know whether your code is just an excerpt from a bigger context. But if you just want to call rsync with grabbing the output synchronosly you could use the following:

my $rsync_base_cmd = "rsync -avm -L --exclude='S/' --include='*/' --in
+clude='*.cpp' " .
     "--include='*.hpp' --exclude='*'";
  
my $command = "$rsync_base_cmd ${login}:${remoteDirectory} $localDirec
+tory";
open my $fh, '-|', $command
    or confess("_copy_component Cannot perform rsync : $!");
my $output;
while(<$fh>) {
    $output .= $_;
}
close $fh or confess("rsync problem: $!");

Best regards
McA

P.S.: untested

[Superfox il Volpone]

Hi McA,
thanks for your reply.

Well, I do not wish to necessarily grab the output from the pipe, the statement while(<$fh>) { $output .= $_; } was a trick to delay the premature death of the pipe.
I am using open3 because the remote server prints into the stderr a welcome message, which it is automatically forwarded into the client stdout using the common pipe mechanism, while I want to get rid of it.

Kind regards,
Superfox

[Anonymous Monk]

But... you are closing the pipe yourself before the process has a chance to terminate: the close() above the waitpid().

If you want to get rid of stderr, simply redirect it. There's no need to mess with open3().

[Superfox il Volpone]

Hi,
I tried to swap waitpid & close, it still closes the process prematurely.

Kind regards,
Superfox