Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change

Re^2: Bidirectional IPC with Expect and Passthrough

by steve (Deacon)
on May 07, 2011 at 13:17 UTC ( #903553=note: print w/replies, xml ) Need Help??

in reply to Re: Bidirectional IPC with Expect and Passthrough
in thread Bidirectional IPC with Expect and Passthrough

IPC::Open2 may also work. IPC::Open3 is not necessary for my application as described in Bidirectional Communication with Another Process:

"There's also an open3() for tridirectional I/O so you can also catch your child's STDERR, but doing so would then require an awkward select() loop and wouldn't allow you to use normal Perl input operations."

The difficulty here is that the conversation follows a slightly different path - user input and program output may occur simultaneously.

Perhaps an a more specific example would be useful. Let's say that I want to make a wrapper for top which allows me to receive the output of the top program (as regularly as possible) and also allows for me to parse the output and if a particular listing is present, return a keystroke/string automatically to top to switch the view so that that process is more easily noticed.

So with output such as:

PID COMMAND %CPU TIME 6555 top 1.5 00:01.42 6537 bash 0.0 00:00.04
no automated action would need to be performed, whereas with this output:
PID COMMAND %CPU TIME 6555 top 1.5 00:02.42 6591 nasty_script 9.9 00:01.01 6537 bash 0.0 00:00.04
would automatically send a string to top to update the view to be more like:
PID COMMAND %CPU TIME 6591 nasty_script 9.7 00:01.11 6555 top 1.1 00:02.52 6537 bash 0.0 00:00.04

The crux of this issue is that user input and program output may occur at any time. I am not aware of a method in perl that allows for receiving data from two sources simultaneously.

With that being stated, I do not need instant reads/writes from either the program output or the user input. What I do need is a way to not lose any potential data in between. If it takes a few milliseconds to read user input, and the program has output during that time I will need to read that after the user input. I would also like to avoid losing keystrokes for user input while the script is reading/checking program output.

Perhaps this would also require a fork so one child could always read/write from/to the program, and the other would always read/write from/to the user?

Replies are listed 'Best First'.
Re^3: Bidirectional IPC with Expect and Passthrough
by BrowserUk (Pope) on May 07, 2011 at 15:19 UTC
    Perhaps this would also require a fork so one child could always read/write from/to the program, and the other would always read/write from/to the user?

    T'is easier using threads.

    I don't have top or anything I can easily substitute for it in an example, but what you are asking for should be relatively trivial using threads.

    You have two distinct, and extremely simple loops you need to perform:

    1. Read whatever comes doen the pipe from the child process and print it to the terminal.
      print while <$fromKid>;
    2. Read anything that the user types and forward it to the child process via the pipe:
      print $toKid while <STDIN>;

    But, as you say, the problem is you need to do both at the same time.

    There are two traditional ways of approaching this problem.

    • A select loop.

      Where you loop around in a busy loop asking is there anything to do? And O I got something, now if it is this do that, or if it is that do something else etc.

      But select loop processing doesn't really play well with buffered IO, so you have to use sysread to avoid blocking. But that means that you are now responsible for doing your own line buffering. And if you have multiple sources, then you have to juggle multiple lines buffers.

      And then there is the problem of dealing with the situation where some of the processing in response to some input takes longer than you can afford to ignore new input, so you have to break that processing up into chunks and that means storing global state to decide what you;ve already done and what else need to be done.

      All very messy and nasty and easy to get wrong.

    • Fork a different process to deal with each source of input and pipe the results back to a parent.

      Except that all you've done is move the goal posts. Ultimately, if the parent is going to control the child, it still needs to monitor and respond to multiple inputs and you are back to needing a select loop with all its inherent problems.

    Now consider a threaded solution:

    #! perl -sw use strict; use threads; use IPC::Open2; my( $toKid, $fromKid ); my $pid = open2( $fromKid, $toKid, 'not-top' ); async{ print while <$fromKid>; }->detach; print $toKid while <STDIN>; kill 9, $pid

    And that is pretty much it. Some extra stuff to handle closing things down and possible errors. But essentially, exactly what you'd like to use. Two loops that run concurrently. Simplicity incarnate.

    Note: That almost certainly won't work with top as is. The reason is that top uses non-blocking, delimiter-less input for its commands, and that doesn;t work via a pipe. Basically, pipes buffer their input and only pass it on to the other end once they've accumulated a buffer full. With a (typically) 4096 character buffer, you'd have to send 4096 single character commands before top would see the first one, and then it would see all 4096 in quick succession and the result would be very confused.

    So, if the program you actually want to control is top, or any program that does delimiter-less input for commands, then you are probably out of luck--using open2 or expect or anything else. But, if you want to control a program that takes normal line-oriented input for commands, then threading is far simpler than other approaches.

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://903553]
and the fire pops...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (3)
As of 2017-07-23 15:42 GMT
Find Nodes?
    Voting Booth?
    I came, I saw, I ...

    Results (347 votes). Check out past polls.