Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

open3 but for n file descriptors

by tange (Initiate)
on Jul 19, 2013 at 16:46 UTC ( #1045381=perlquestion: print w/ replies, xml ) Need Help??
tange has asked for the wisdom of the Perl Monks concerning the following question:

open3 only takes STDIN, STDOUT and STDERR. If the command being run uses other file descriptors then open3 cannot capture those:

echo foo # Can be captured echo foo >&2 # Can be captured echo foo >&3 # Cannot be captured (with open3)
I have looked into IPC::Run::run which seems to be able to deal with that, but I also need the PID and I found no way for IPC::Run::run to give me that. Also IPC::Run is not installed on many of the old distributions supported by GNU Parallel, whereas open3 is.

How can I wrap open3 so that I can capture file descriptors 3..61 in the same way I can capture STDOUT and STDERR?

Background

This is intended for a possible extension of GNU Parallel, so you can do:

# Currently mixes output due to fd 3 not buffered parallel 'echo {} start >&3;sleep 10;echo {} end >&3' ::: a b c 3> +out.file
without having the output from different jobs mixed. Right now only STDOUT and STDERR are buffered and thus guaranteed not to mix.
# This currently works parallel 'echo {} start >&2;sleep 10;echo {} end >&2' ::: a b c 2> +out.file

Comment on open3 but for n file descriptors
Select or Download Code
Re: open3 but for n file descriptors
by Loops (Curate) on Jul 19, 2013 at 19:32 UTC

    Are you sure you really need to have the PID of child processes when using IPC::Run? It allows you to send signals and kill children directly. If you really must have the PID, it's possible to use start/pump/finish instead of run:

    use strict; use warnings; use IPC::Run qw( start ); # Initiate the child process my $data; my $s = start [ 'ls' ], '>', \$data; # Show its PID for my $kid ( @{$s->{KIDS}} ) { print $kid->{PID}; } # pump data in and out, then clean up $s->pump; $s->finish;

      As mentioned I would prefer not to use IPC::Run because IPC::Run is not installed on many of the old distributions supported by GNU Parallel, whereas open3 is. There is typically a reason why people do not upgrade the old systems, and it is therefore unlikely that IPC::Run will be installed.

      The PID is currently the way I keep track of which process was started when, so when using --keep-order, I can make sure the output from the dead children is printed in the correct order. But with a bit of recoding this could probably be changed.

      I had hoped wrapping open3 with more filedescriptors would be as easy as this meta-code for fd3:

      # Record fd3 for later use open($orig_fd{3},">&=",3); for $id (1..10) { open($child{$id,3},">",$tmpfile{$i}); # make $child{$id,3} file descriptor 3 for me and all future childre +n connect_filedescriptor(3,$child{$id,3}); # start the child open3(...) } sub reaper { # When a child dies seek $child{$id,3},0,0; # Print the captured fd3 to the parent's original fd3 print $orig_fd{3} <$child{$id,3}>; close $child{$id,3}; }
      I know how to make most of the meta-code, but connect_filedescriptor I do not know how to make.

        Looking at the man page for open, it seems what I need is this:
        open(my $err, '>', "foo.out"); open(STDERR, ">&", $err);
        but in a generic version, so I can use other file descriptors than STDERR (fd2).

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1045381]
Approved by hdb
Front-paged by Old_Gray_Bear
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (5)
As of 2015-07-07 03:09 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (86 votes), past polls