Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Re: IPC::Open3 woes

by abstracts (Hermit)
on Mar 15, 2002 at 01:59 UTC ( [id://151886]=note: print w/replies, xml ) Need Help??


in reply to IPC::Open3 woes

Hello all,

BazB asked me to post a complete example of getting IPC::Open3 to work with IO::Select. Please comment on the code if you see any problems with it.

Thanks a million.

Aziz,,,

############################################################ #!/usr/bin/perl -w # this script printf to stdout and stderr. It prints random # characters and does not flush the output of stdout. stderr # is autoflushed by default. # uncomment the line about autoflush STDOUT to see how that # changes the behavior. Also, you can uncomment the sleep # line to watch the script in slow motion. use warnings; use strict; use IO::Handle; #autoflush STDOUT 1; for (1..100){ my $str = ''; for(1..80){ $str .= ('A'..'Z','a'..'z',0..9)[rand 62]; } if(int rand 2){ # 50:50 chance print STDOUT "StdOut:$str"; } else { print STDERR "StdErr:$str"; } # sleep 1; } ############################################################ #!/usr/bin/perl -w # this script runs the previous script and catches both # child's stdout and stderr. It prints progress information # as it goes. It also prints the data it gets to the # corresponding file. use strict; use warnings; #use diagnostics; use IPC::Open3; use IO::Select; use Symbol; my $cmd = "./test.pl"; open(ERRLOG, ">error.log") or die "Can't open error log! $!"; open(OUTPUT, ">output.log") or die "Can't open output log! $!"; my ($infh,$outfh,$errfh); $errfh = gensym(); # if you uncomment this line, $errfh will # never be initialized for you and you # will get a warning in the next print # line. my $pid; eval{ $pid = open3($infh, $outfh, $errfh, $cmd); }; die $@ if $@; print "IN: $infh OUT: $outfh ERR: $errfh\n"; print "PID was $pid\n"; # now our child is running, happily printing to # its stdout and stderr (our $outfh and $errfh). my $sel = new IO::Select; # create a select object $sel->add($outfh,$errfh); # and add the fhs # $sel->can_read will block until there is data available # on one or more fhs while(my @ready = $sel->can_read) { # now we have a list of all fhs that we can read from foreach my $fh (@ready) { # loop through them my $line; # read up to 4096 bytes from this fh. # if there is less than 4096 bytes, we'll only get # those available bytes and won't block. If there # is more than 4096 bytes, we'll only read 4096 and # wait for the next iteration through the loop to # read the rest. my $len = sysread $fh, $line, 4096; if(not defined $len){ # There was an error reading die "Error from child: $!\n"; } elsif ($len == 0){ # Finished reading from this FH because we read # 0 bytes. Remove this handle from $sel. # we will exit the loop once we remove all file # handles ($outfh and $errfh). $sel->remove($fh); next; } else { # we read data alright print "Read $len bytes from $fh\n"; if($fh == $outfh) { print OUTPUT $line; } elsif($fh == $errfh) { print ERRLOG $line; } else { die "Shouldn't be here\n"; } } } } # now that the child closed both its handles, I assume it # exited. # ps will show you the <defunct> child. print `ps`; # go ahead and reap it waitpid $pid, 0; # wait for it to die # not it's no more print `ps`; close(ERRLOG) or die "Can't close filehandle! $!"; close(OUTPUT) or die "Can't close filehandle! $!";

Replies are listed 'Best First'.
Re^2: IPC::Open3 woes
by LNEDAD (Novice) on Mar 24, 2010 at 13:15 UTC

    I am hopeing that somebody can add some clarity to this old thread. Can somebody tell me why the following coded hangs if the line which defines the @outlines array is commented out?

    I have used open3 in other places w/o reading the the child's output and the process didn't hang. I expect it is because in that case the command didn't have output.

    I understand how some of the other example programs in this thread prevent locking while read from open3's handles but it seems strange that this would lock when I am not reading from them and just calling waitpid as soon as possible.

    Also, should I explicitly close $wtr, $rtr, and $err?

    ... my($wtr,$rdr); my $err = gensym; ### $logcmd is a command that returns approx. 460 lines. my $pid = open3($wtr, $rdr, $err, "$logcmd"); my @outlines = <$rdr>; ### if this line is commented out we hang on w +aitpid waitpid( $pid, 0 ); ...

      I am hopeing that somebody can add some clarity to this old thread. Can somebody tell me why the following coded hangs if the line which defines the @outlines array is commented out?

      You have a deadlock.

      • waitpid never returns because the child never exits.
      • The child never exits because it's blocked trying to write a full pipe.
      • The pipe is full because noone is emptying it.

      If it doesn't happen for all your children, it's because they don't output enough data to fill the pipe.

      use strict; use warnings; use IPC::Open3 qw( open3 ); for (my $size = 1024; ; $size += 1024) { print("$size\n"); my $pid = open3( local our $to_chld, local our $fr_chld, local our $fr_chld_err, perl => ( -e => 'print "x" x $ARGV[0]', $size ) ); waitpid($pid, 0); }
      ... 62464 63488 64512 65536 66560 [hangs]^C

      The solution is to redirect the output to /dev/null

      use strict; use warnings; use IPC::Open3 qw( open3 ); open(local *DEVNULL, '>', '/dev/null') or die; for (my $size = 1024; ; $size += 1024) { print("$size\n"); my $pid = open3( local our $to_chld, '>&DEVNULL', '>&DEVNULL', perl => ( -e => 'print "x" x $ARGV[0]', $size ) ); waitpid($pid, 0); }
      ... 365568 366592 367616 368640 369664 370688 371712 372736 373760 374784 ...
      Please somebody do comment, this is great information. I fear it is now out of date. I can't get this to work under Perl 5.8 any more. The documentation does not help. The code hangs on the open3 call.
Re: IPC::Open3 woes
by polesworth (Initiate) on Oct 09, 2003 at 14:01 UTC
    Hi,

    I've been trying your example, and it works great on linux (perl 5.6.1), but doesn't work on win2k (activestate perl 5.8.0). I've managed to track it down as far as the line where you call can_read, which for some odd reason appears to cause the loop to exit immediately (probably returning undef). Calling count() immediately before the can_read shows that there are still 2 file handles.

    I'm a bit stuck on this one, so if anyone has any ideas, they'd be appreciated.

    Thanks.

    Miah.

      Hi, I am new to open3 and perldoc is too vague to understand
      I am trying a simple unix command to run in open3.
      open3($stdin, $stdout, $stderr, "gcp -vpfR $xrc $dst")


      What i need is -- I want to caputre STDOUT until gcp finishes or there is something on STDERR (on disk full case etc).

      while (<$stdout>) {
      print "$_";
      }
      @stderr = <$stderr>;
      if (@stderr) { dir "ERROR: gcp\n" }


      BUT this is not working -- any suggestions

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://151886]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others perusing the Monastery: (4)
As of 2024-04-19 14:40 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found