How to change Open2 input buffering

[thecap]

I am using Open2 to run a C program that outputs a line with a timestamp in reply to each input line. The problem is that the reply lines are buffered together.

Desired results (happens when program is run from terminal or if C program calls fflush(stdout) after each printf):

Input1
time(): Output1
END
time(): exit

Observed with Open2 and no fflush in the C program:

Input1
END
time(): Output1
time(): exit

I believe the problem is that C program's libc tries to detect if stdout is connected to a terminal. If it is connected to a terminal it uses line buffering, otherwise it uses full buffering. How can I make the stdout of a program started with Open2 appear to be connected to a terminal?

I do not want to change the source of the C program. I am using IO::Select to see if there is any output as the actual program I want to use will only sometimes produce output.

I tried looking at the glibc-2.3.2 source to find how it decides what buffering to use on stdout, but could not work out how the line "./sysdeps/standalone/open.c: setvbuf( stdout, NULL, _IOLBF, BUFSIZ );" is called.

Here is a strace dump of running the C program with Open2:

read(0, "Input1\n", 4096)               = 7
time(NULL)                              = 1066404185
read(0, "END\n", 4096)                  = 4
time(NULL)                              = 1066404190
write(1, "1066404185: \"Input1\"\n1066404190:"..., 38) = 38

Here is a run with the same inputs from the shell:

read(0, "Input1\n", 1024)               = 7
time(NULL)                              = 1066404693
write(1, "1066404693: \"Input1\"\n", 21) = 21
read(0, "END\n", 1024)                  = 4
time(NULL)                              = 1066404697
write(1, "1066404697: exit\n", 17)      = 17

Thank you for any ideas you may have to help me,
Tom

[tye]

This has been a known problem for well over a decade. There are several ways to fix this that would be trivial to implement. Probably the best place to start is to get GNU to add such a feature to their C RTL so that others might follow suit.

If you have access to your C RTL source code and can build a new shared library, then you can even fix this yourself (even for C programs that you don't have the source code to -- unless they were statically linked).

The simplest fix I'd use would be an environment variable that, when set, tells the C RTL to act like isatty(2) returned true for stdout (etc.) and so would use "line buffering" instead of "full buffering" (for example, export LINEBUFFEREDPIPES=1).

The current "solution" is to use pseudo ttys instead of pipes. This is quite the aircraft carrier to swat a fly, but that gives you another route to investigate.

[ptkdb]

Another solution in the brute force category, is there any input that you can give to the C program that will get it to produce output, thereby pushing data out of the buffer?

You may just have to bite the bullet and change the program.

At a guess, you're trying to solve some production issue problem with perl script, but the the C program in question belongs to some other group, is legacy code, or is under "Ivory Tower Source Control", or some other situation that makes modifying it, even with just 1 line of code, difficult.

I know that in certain organizations modifying a legacy program is tougher than passing a bill through Congress, but if they have any kind of tracking system, such as ClearQuest, file the 'Request for an Enhancement', or even elevate it to the level of a bug on the basis of "failure to flush output, prevents programs use with automated scripting tools."

[sgifford]

You need to use a pseudo-tty to solve this problem. IO::Pty is a fairly easy way to solve this. Here's an example:

#!/usr/bin/perl -w

use strict;

use IO::Pty;

my $pty = IO::Pty->new
    or die "Pty error: $!\n";
my $slave = $pty->slave
    or die "Pty slave error: $!\n";

# Run cat with stdout on the PTY.
pipe(PIPEREAD,PIPEWRITE);
if (fork()==0)
{
  # Child
  close(PIPEWRITE);
  close($slave);
  close(STDOUT)
      or die "Couldn't close STDOUT: $!\n";
  open(STDOUT,">&".$pty->fileno)
      or die "Couldn't re-open STDOUT: $!\n";
  close(STDIN)
      or die "Couldn't close STDIN: $!\n";
  open(STDIN,"<&PIPEREAD")
      or die "Couldn't re-open STDIN: $!\n";
  exec("/bin/cat")
      or die "Couldn't exec /bin/cat: $!\n";
}

# Parent
close(PIPEREAD);
close($pty);

select(PIPEWRITE);
$|=1;
select(STDOUT);

warn "Parent running\n";

foreach my $val (1..10) {
  print PIPEWRITE "$val\n";
  $_ = <$slave>;
  print "$_";
}

[thecap]

Thank you tye, ptkdb, and sgifford for your excellent replies.

The pseudo-tty code sgifford posted worked wonderfully in Linux, but not when I moved to my final target, SunOS. I eventually found a small C program called ircflush, part of the ircII distribution that does the same thing and seems to work in SunOS. I also looked into using IPC::Run but it does not connect file handles to pseudo-ttys.

[sgifford]

Huh. What version of SunOS?