Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

buffering when reading from unnamed pipe with child process

by Santasha (Initiate)
on Nov 19, 2020 at 21:49 UTC ( #11123851=perlquestion: print w/replies, xml ) Need Help??

Santasha has asked for the wisdom of the Perl Monks concerning the following question:

This code...

#!/usr/bin/perl use strict; use warnings; use IO::Pipe; use IO::Handle; use IO::Select; my $pipe = IO::Pipe->new(); my $pid = fork(); die "failed to fork: $!" unless defined $pid; if ($pid) { $pipe->reader(); my $select = IO::Select->new(); $select->add($pipe); while (1) { my @ready = $select->can_read(1); foreach my $h (@ready) { my $line = <$h>; chomp $line; print "PARENT sees <$line>\n"; } } } else { $pipe->writer(); $pipe->autoflush(1); my $c = 0; while (1) { $c++; print "CHILD writing <$c>\n"; $pipe->print("$c\n"); sleep 1 unless $c % 5; # pause after each fifth line } }

produces output like this...

CHILD writing <1> CHILD writing <2> CHILD writing <3> CHILD writing <4> CHILD writing <5> PARENT sees <1> CHILD writing <6> CHILD writing <7> CHILD writing <8> CHILD writing <9> CHILD writing <10> PARENT sees <2> PARENT sees <3> PARENT sees <4> PARENT sees <5> PARENT sees <6> CHILD writing <11> CHILD writing <12> CHILD writing <13> CHILD writing <14> CHILD writing <15> PARENT sees <7> PARENT sees <8> PARENT sees <9> PARENT sees <10> PARENT sees <11>

And so on.

What do I need to do to avoid the buffering of child writes that seems to be occuring?

This is on Debian/Linux.

Replies are listed 'Best First'.
Re: buffering when reading from unnamed pipe with child process
by tybalt89 (Prior) on Nov 19, 2020 at 23:05 UTC
    #!/usr/bin/perl use strict; # https://perlmonks.org/?node_id=11123851 use warnings; use IO::Pipe; use IO::Handle; use IO::Select; my $pipe = IO::Pipe->new(); my $pid = fork(); die "failed to fork: $!" unless defined $pid; if ($pid) { $pipe->reader(); my $select = IO::Select->new(); $select->add($pipe); my $line = ''; while (1) { my @ready = $select->can_read(1); foreach my $h (@ready) { # my $line = <$h>; if( sysread $h, $line, 8192, length $line ) { # chomp $line; while( $line =~ s/(.*)\n// ) { print "PARENT sees <$1>\n"; } } else { close $pipe }; } } } else { $pipe->writer(); $pipe->autoflush(1); my $c = 0; while (1) { $c++; print "CHILD writing <$c>\n"; $pipe->print("$c\n") or die; sleep 1 unless $c % 5; # pause after each fifth line } }
    CHILD writing <1> CHILD writing <2> CHILD writing <3> CHILD writing <4> CHILD writing <5> PARENT sees <1> PARENT sees <2> PARENT sees <3> PARENT sees <4> PARENT sees <5> CHILD writing <6> CHILD writing <7> CHILD writing <8> CHILD writing <9> CHILD writing <10> PARENT sees <6> PARENT sees <7> PARENT sees <8> PARENT sees <9> PARENT sees <10> CHILD writing <11> CHILD writing <12> CHILD writing <13> CHILD writing <14> CHILD writing <15> PARENT sees <11> PARENT sees <12> PARENT sees <13> PARENT sees <14> PARENT sees <15>

      Greatly appreciated, thank you.

        I have found setting autoflush in the parent via $|++; also helps; but using the pipe interface directly it also makes sense to do it this way. My recommendation in addition to the explicit flush is to set autoflush in the parent.
Re: buffering when reading from unnamed pipe with child process
by GrandFather (Sage) on Nov 19, 2020 at 23:29 UTC

    After the fork you have two processes which each get slices of time in which to do some work. That means that the nature of that work is bursty - busy for a while, idle for a while - depending on what else is going on in the machine so for both producer and consumer the processing is in bursts and that is essentially out of the control of the processes. It's not the pipe that is introducing the burstiness through buffering, but it is a fundamental property of preemptive multi tasking operating systems.

    You could introduce semaphores or mutexes to lock step the two processes, but then you probably lose the advantage of having two processes anyway.

    Update: that might make sense if the OP didn't have a sleep generating output in bursts. See tybalt89's reply below.

    Optimising for fewest key strokes only makes sense transmitting to Pluto or beyond

      Nope.

      The first buffered read <$h> grabs all five of the first sent lines, and leaves the handle empty, thus the delay for the second read. It's not until the 6th line comes in that the program can get to the second read.

      The usual advice is to not mix buffered and un-buffered I/O, and this code is the perfect example of why. IO::Select operates on unbuffered data while readline is buffered.
      That's why I used sysread (which is unbuffered) rather than readline.

      NOTE: Using sysread may get you more than one line at a time (or even partial lines) which is why I added the loop that extracts and deletes one while line at a time.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://11123851]
Approved by GrandFather
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (7)
As of 2020-12-03 08:31 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    How often do you use taint mode?





    Results (53 votes). Check out past polls.

    Notices?