Getting Call-Response Behavior with Expect

[gnosti]

I would like to talk to an external program (midish) from perl using Expect (or Expect::Simple). I want to spawn this program once, keeping the child process loaded while I do other stuff. AFAIK Midish is well-behaved, using STDIN and STDOUT for I/O. However I'm having a hard time getting a reponse reliably after sending a command.

Update 1: I will investigate approaches other than Expect.

Update 2: Zentara's solution using IPC::Open3 is *way* simpler.

My first experiment was a simple command loop:

- get a prompt ("+ready")

- send a command

- get a reply

I thought I would try with Expect::Simple, however some buffering or command-ordering problems arise.

------ test-expect-simple ----

use Modern::Perl;
use Expect::Simple;

$| = 1;
my $obj = Expect::Simple->new({
     Cmd => [ midish => '-v'],
     Prompt => '+ready',
     DisconnectCmd => 'exit',
     Verbose => 0,
     Debug => 0,
     Timeout => 100
   });

sub prompt { print "midish> " }


while ( prompt(), my $cmd = <STDIN> ){
    
        exit if $cmd =~ 'quit';
        chomp $cmd;
        $cmd .= "\r";
        $obj->send( $cmd );
        my @lines = split "\n", $obj->before;
        splice(@lines, 0, 2); # throw away the first two lines
        say join "\n", @lines;
}

__END__

$perl test-expect-simple
midish> asdf

asdf: no such proc
midish> 1234

midish> ffff

3.5: statement or proc definition expected
midish> ^C

Comment: I'm not getting the full message after sending a command. Maybe it's a problem with Expect::Simple.

So here is the next try, a command loop using Expect.

------ test-expect-loop -----

use Expect;
use Modern::Perl;

my $exp = Expect->spawn("midish","-v")
    or die "Couldn't start program: $!\n";

# don't copy program output to STDOUT
$exp->log_stdout(0);

$exp->expect(1, '+ready', \&do_cmd);

sub do_cmd {

    map{say "midish: $_"} split "\n", $exp->before;
    print "enter command >> ";
    my $cmd = <STDIN>;
    $exp->send($cmd);
    exp_continue;
}

$exp->soft_close();

__END__

program output:


$ perl test-expect-loop
enter command >> adsfsd
midish: 
midish: adsfsd
midish: adsfsd: no such proc
enter command >> 1234
midish: 
midish: 1234
midish: 2.5: statement or proc definition expected
enter command >> ffff
midish: 
midish: ffff
midish: ffff: no such proc
enter command >> exit
$

Good, I'm getting output after every command. But my application is structured more like this:

call_to_expect($one_command);
execute_other_code();
call_to_expect($another_command);

------ test-expect-unrolled -----

use Expect;
use Modern::Perl;

my $exp = Expect->spawn("midish","-v")
    or die "Couldn't start program: $!\n";

# don't copy program output to STDOUT
$exp->log_stdout(0);

$exp->expect(1, '+ready', \&do_cmd);
print_output();

$exp->expect(1, '+ready', \&do_cmd);
print_output();

$exp->expect(1, '+ready', \&do_cmd);
print_output();

sub do_cmd {

    print "enter command >> ";
    my $cmd = <STDIN>;
    $exp->send($cmd);
}
sub print_output {
    map{say "midish: $_"} split "\n", $exp->before;
}

$exp->soft_close();
__END__

program output:

$ perl test-expect-unrolled
enter command >> asdf
enter command >> 1234
midish: 
midish: asdf
midish: asdf: no such proc
enter command >> ffff
midish: 
midish: 1234
midish: 2.5: statement or proc definition expected
<hang> 
^C
$

So, I still have a problem getting a one-to-one correspondence between the command string and the result string.

Any ideas how I could make one call to send a command and get a response, in a way that I can repeat indefinitely? Thanks for reading!

[zentara]

You may be able to use IPC::Open2 or IPC:Open3 to simplify your code.

For example, try running this:

#!/usr/bin/perl
use warnings;
use strict;
use IPC::Open3;
use IO::Select;

my $pid = open3(\*WRITE, \*READ,\*ERROR,"/bin/bash");

my $sel = new IO::Select();

$sel->add(\*READ);
$sel->add(\*ERROR);

my($error,$answer)=('','');

while(1){

 print "Enter command\n";
 chomp(my $query = <STDIN>);
 
 #send query to bash
 print WRITE "$query\n";

 foreach my $h ($sel->can_read)
 {
   my $buf = '';
   if ($h eq \*ERROR)
   {
     sysread(ERROR,$buf,4096);
     if($buf){print "ERROR-> $buf\n"}
   }
   else
   {
     sysread(READ,$buf,4096);
     if($buf){print "$query = $buf\n"}
   }
 }
}

waitpid($pid, 1);
# It is important to waitpid on your child process,  
# otherwise zombies could be created.

[gnosti]

Zentara,

That's helpful snippet. Just from running that code with the target CL app, I see I wasn't distinguishing between output that came via STDERR and STDOUT in my Expect example.

I read about IPC::Open2/3 before, and briefly gandered IPC::Run, so I don't know what I got waylaid by Expect.

I guess I expected something more... Also, the 'select' call is something new to me. Worth learning about.

[Proclus]

Obviously you have invested some time on Expect so I do not want to steer you to a different solution but just to keep in mind, POE offers excellent solutions to these kind of problems.

You can have a look at POE::Wheel::Run . It will allow you to register callbacks for STDIN and STDOUT so that you can talk to your process in a nonblocking fashion.

[gnosti]

Long enough to be discouraged by its approach. If anything, I'm more inclined to look elsewhere. I'd thought that Expect was a well-established way to address this problem space.

My application does use an event framework already. I have AnyEvent, with Event or Tk underneath, depending on if the user starts the app in GUI mode.

I'm not especially keen to switch out event frameworks in order to do a little IO with a command-line utility extending my app. But I will investigate POE.

[Proclus]

In addition to your comments, POE::Wheel::Run may give you hard time in a GUI environment esp on Windows.
I haven't used AnyEvent myself, but I've seen that it can be used with POE.