Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

Re: Redirecting stdout/stderr to pipe

by Thelonius (Priest)
on Sep 19, 2005 at 19:36 UTC ( #493274=note: print w/replies, xml ) Need Help??


in reply to Redirecting stdout/stderr to pipe

I think your main problems is that select does not interact well with buffered I/O, which you use with <READERR> and <READOUT>. You should really use sysread and do a split /\n/ at the very end.

Here's how I might do it, using IPC::Open3 and IO::Select. You should add the waitpid/close from your example:

#!perl -w use IPC::Open3; use IO::Select; use Symbol qw(gensym); use strict; my $cmd = shift; my $g_timeout = shift || 5; my $g_maxlines = shift || 10; my @output; my @err; $| = 1; runcmd(\$cmd, \@output, \@err); print "output = \n"; print " $_\n" for @output; print "error = \n"; print " $_\n" for @err; sub runcmd { my ($cmdref, $outref, $errref) = @_; my ($childin, $childout, $childerr); $childerr = gensym; my $pid = open3($childin, $childout, $childerr, $cmd) or die "Cannot run cmd '$cmd': $!\n"; my $select = IO::Select->new or die "Cannot create select object: $! +\n"; my @hold_output; for (($childout, $childerr)) { $select->add($_); $hold_output[fileno($_)] = [0, 0, ""]; # eof, lines, buffer; } $@ = undef; eval { local $SIG{ALRM} = sub { die "alarm\n" }; my $deadline = $g_timeout + time; my $g_stop = 0; alarm($g_timeout + 1); while (!$g_stop && $select->count > 0) { $! = 0; my @ready = $select->can_read($deadline - time); if (!@ready) { $g_stop = 1; } for my $handle (@ready) { my $fno = fileno($handle); my $line; my $bytesread = sysread $handle, $line, 1024; if ($bytesread) { $hold_output[$fno][2] .= $line; $hold_output[$fno][1] += $line =~ y/\n/\n/; if ($hold_output[$fno][1] >= $g_maxlines) { $select->remove($handle); } } elsif ($!) { die "$!"; } else { $hold_output[$fno][0] = 1; #EOF $select->remove($handle); } } } alarm(0); }; if ($@) { # print STDERR "\$\@ = $@\n"; die unless $@ eq "alarm\n"; } # Note: lines may exceed $g_maxlines because of buffering # of output in the child process @$outref = split /\n/, $hold_output[fileno($childout)][2]; @$errref = split /\n/, $hold_output[fileno($childerr)][2]; }

Replies are listed 'Best First'.
Re^2: Redirecting stdout/stderr to pipe
by 0xbeef (Hermit) on Sep 21, 2005 at 13:17 UTC
    Okay, thanks for that! I would like however like to be enlightened more: ;)

    1. I'd rather not use a solution using external modules, since the collector will have to run on MANY hosts as painlessly and non-intrusively as possible. Is there a high performance approach without external modules?
    2. I am unsure as to how much data should be read from the pipe at a time in order to optimise throughput as much as possible...

    -0xbeef

      1. I'd rather not use a solution using external modules, since the collector will have to run on MANY hosts as painlessly and non-intrusively as possible. Is there a high performance approach without external modules?
      The modules cited in the code that Thelonius posted (IPC::Open3, IO::Select, Symbol) are not "external", in the sense that they are all included in the standard Perl "core" distribution. That is, wherever a reasonably current version of Perl is installed, these modules are also installed by default.

      (If these many hosts you speak of have non-standard or hopelessly outdated perl installations, that's going to be a problem anyway.)

      As for buffer size, the 1024 bytes suggested by Thelonius is fine for handling text that must ultimately be treated in a line-oriented fashion; if you're dealing with really high data rates, an 8k buffer should be about optimal.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://493274]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (3)
As of 2022-09-26 03:05 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    I prefer my indexes to start at:




    Results (116 votes). Check out past polls.

    Notices?