Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

open (to read) and kill a pipe to a pipe

by Anonymous Monk
on Aug 26, 2010 at 20:19 UTC ( #857561=perlquestion: print w/ replies, xml ) Need Help??
Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hi there!

So I need to read the output of a command that outputs a lot of data... but I don't need all the data. I want to be able to stop after some point.

For just a single pipe to my script this works OK:

$pid = open (INPUT, "mycommand $somefile |") or die ...; while (<INPUT>) { if ( ... trigger to stop ... ) { kill 9, $pid; last; } } close INPUT;

At the point the kill is issued, mycommand is terminated, life is good and we are ready to move on and process the next file.

... by the way, I'm on Windows XP, ActiveState Perl 5.8.8.

I'm not even sure what signal I should be sending with the kill (I tried 1, 2 and 9... 9 worked so that's what I am using).

What I want to do is:

$pid = open (INPUT, "gzip -dc $somegzfile | mycommand |") or die ...; while (<INPUT>) { if ( ... trigger to stop ... ) { kill 9, $pid; last; } } close INPUT;

Notice the pipe to a pipe... "gzip ... | mycommand |"

This works in the sense that it gunzips the file on the fly, and passes the data to mycommand, and then I get what I want in the script.

BUT it does not kill the gzip or mycommand processes when it gets to the "kill 9, $pid". Then it just sits at the close INPUT until the gzip and mycommand are done (which takes a few minutes).

Any ideas on how I should do the piping so I can kill the processes properly? Also, any suggestions on what signals to use in windows?

Thanks!
David A.

Comment on open (to read) and kill a pipe to a pipe
Select or Download Code
Re: open (to read) and kill a pipe to a pipe
by darist (Initiate) on Aug 26, 2010 at 20:26 UTC
    -- I posted the question... but somehow got logged off... by the way... How do I sign up for email alerts when someone replies to a given post?
      You don't, AFAIK.

      But you *CAN* set your Message Settings to receive a message-alert (in the CB sidebar).

Re: open (to read) and kill a pipe to a pipe
by ikegami (Pope) on Aug 26, 2010 at 21:07 UTC

    I'm not even sure what signal I should be sending with the kill

    Windows has two signals, Ctrl-C and Ctrl-Break. It uses message passing primarily, but that's for application with Windows. It can terminate processes forcibly.

    I think you can send the Ctrl-C and Ctrl-Break signals using kill, but I don't know which numbers do this.

    kill 9 surely results in the the system call to terminate a process. That's drastic.

    BUT it does not kill the gzip or mycommand processes when it gets to the "kill 9, $pid".

    It kills the shell whose pid is in $pid. This doesn't kill the shell's children. I don't know one would achieve that.

    One solution would be to eliminate the shell and do the piping yourself.

      Thanks! This looks promesing... but I can't figure it out.

      I tried open2 (since I don't need the stderr from open3), but I don't even get the output of mycommand to show up...
      what should I put in the while(< ??? >)... or did I even use the open2 command properly?
      use IPC::Open2; $file = $ARGV[0]; local *FRGZ; local *TOGZ; local *FRCMD; $pid1 = open2(*FRGZ, *TOGZ, "gzip -dc $file"); $pid2 = open2(*FRCMD, '<FRGZ', 'mycommand'); while (<*FRCMD>) { print; }
      maybe someone can provide an example of how to make this work...
      My idea here is for the command "gzip -dc $file" to pipe its output to *FRGZ. Then I would pipe *FRGZ into "mycommand" and the output would be on *FRCMD .... ???

        open2 mixes STDERR into STDOUT, its arguments are "backwards", and it just ends up calling open3 anyway. I avoid it.

        use strict; use warnings; use IPC::Open3 qw( open3 ); my $file = $ARGV[0]; open(local *TO_CHAIN, '<', $file) or die("Can't open \"$file\": $!\n"); my $gz_pid = open3('<TO_CHAIN, local *PIPE, '>STDERR', 'gzip -dc'); my $mc_pid = open3('<PIPE', local *FR_CHAIN, '>STDERR', 'mycommand'); while (<FR_CHAIN>) { print; }
Re: open (to read) and kill a pipe to a pipe
by JavaFan (Canon) on Aug 26, 2010 at 21:23 UTC
    Why not just close the file handle?

      Which file handle, the one from which gzip reads that has nothing to do with Perl? Well, you could pipe information to gzip from Perl, so you're back to using open3 or the like. Additionally, you need to feed information to gzip while reading from mycommand, so you need some kind or parallisation. You could sockify the pipes and use select or you could use threads.

      It's getting complicated fast, and it might not even help enough, depending on how long it takes for mycommmand processes a pipe buffer's worth of data.

Re: open (to read) and kill a pipe to a pipe
by Your Mother (Canon) on Aug 26, 2010 at 21:37 UTC

    Try this minor change-

     kill 9, -$pid;

    I didn't try it on your stuff but it did work on a simple system("thingy &") test I just tried.

      Thanks Your Mother!
      kill 9, -$pid; # didn't work # But... kill -9, $pid # WORKS !!!

      From perldoc on kill:
      if SIGNAL is negative, it kills process groups instead of processes. That means you usually want to use positive not negative signals.
        That's interesting since Windows doesn't have process groups. I wonder what it actually does ( when asked to kill a process group ).
      You said ActivePerl and Windows, but system("thingy &") doesn't work on Windows. ("&" is like sh's "&&".)

      Mixed YourMother's post and the OP's

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://857561]
Approved by kennethk
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (6)
As of 2014-07-25 00:17 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (167 votes), past polls