Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number

open3 but for n file descriptors

by tange (Initiate)
on Jul 19, 2013 at 16:46 UTC ( #1045381=perlquestion: print w/replies, xml ) Need Help??
tange has asked for the wisdom of the Perl Monks concerning the following question:

open3 only takes STDIN, STDOUT and STDERR. If the command being run uses other file descriptors then open3 cannot capture those:

echo foo # Can be captured echo foo >&2 # Can be captured echo foo >&3 # Cannot be captured (with open3)
I have looked into IPC::Run::run which seems to be able to deal with that, but I also need the PID and I found no way for IPC::Run::run to give me that. Also IPC::Run is not installed on many of the old distributions supported by GNU Parallel, whereas open3 is.

How can I wrap open3 so that I can capture file descriptors 3..61 in the same way I can capture STDOUT and STDERR?


This is intended for a possible extension of GNU Parallel, so you can do:

# Currently mixes output due to fd 3 not buffered parallel 'echo {} start >&3;sleep 10;echo {} end >&3' ::: a b c 3> +out.file
without having the output from different jobs mixed. Right now only STDOUT and STDERR are buffered and thus guaranteed not to mix.
# This currently works parallel 'echo {} start >&2;sleep 10;echo {} end >&2' ::: a b c 2> +out.file

Replies are listed 'Best First'.
Re: open3 but for n file descriptors
by Loops (Curate) on Jul 19, 2013 at 19:32 UTC

    Are you sure you really need to have the PID of child processes when using IPC::Run? It allows you to send signals and kill children directly. If you really must have the PID, it's possible to use start/pump/finish instead of run:

    use strict; use warnings; use IPC::Run qw( start ); # Initiate the child process my $data; my $s = start [ 'ls' ], '>', \$data; # Show its PID for my $kid ( @{$s->{KIDS}} ) { print $kid->{PID}; } # pump data in and out, then clean up $s->pump; $s->finish;

      As mentioned I would prefer not to use IPC::Run because IPC::Run is not installed on many of the old distributions supported by GNU Parallel, whereas open3 is. There is typically a reason why people do not upgrade the old systems, and it is therefore unlikely that IPC::Run will be installed.

      The PID is currently the way I keep track of which process was started when, so when using --keep-order, I can make sure the output from the dead children is printed in the correct order. But with a bit of recoding this could probably be changed.

      I had hoped wrapping open3 with more filedescriptors would be as easy as this meta-code for fd3:

      # Record fd3 for later use open($orig_fd{3},">&=",3); for $id (1..10) { open($child{$id,3},">",$tmpfile{$i}); # make $child{$id,3} file descriptor 3 for me and all future childre +n connect_filedescriptor(3,$child{$id,3}); # start the child open3(...) } sub reaper { # When a child dies seek $child{$id,3},0,0; # Print the captured fd3 to the parent's original fd3 print $orig_fd{3} <$child{$id,3}>; close $child{$id,3}; }
      I know how to make most of the meta-code, but connect_filedescriptor I do not know how to make.

        Looking at the man page for open, it seems what I need is this:
        open(my $err, '>', "foo.out"); open(STDERR, ">&", $err);
        but in a generic version, so I can use other file descriptors than STDERR (fd2).

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1045381]
Approved by hdb
Front-paged by Old_Gray_Bear
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others meditating upon the Monastery: (5)
As of 2017-05-27 15:57 GMT
Find Nodes?
    Voting Booth?