Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Detect whether a writeable filehandle has closed?

by jdporter (Canon)
on Jan 13, 2021 at 17:26 UTC ( #11126853=perlquestion: print w/replies, xml ) Need Help??

jdporter has asked for the wisdom of the Perl Monks concerning the following question:

In my perl program I open a subprocess for writing via a pipe. It appears that this child process can decide to exit at times I don't expect. Is it possible to detect that the filehandle to the pipe is no longer usable? As it is, I now get SIGPIPE occasionally, but it seems to happen later, after I've possibly already written (tried to write) additional lines to the pipe. I want something synchronous.

I've seen Scalar::Util's openhandle function. Does it work on pipe handles, opened for writing?

TIA!

I reckon we are the only monastery ever to have a dungeon stuffed with 16,000 zombies.

Replies are listed 'Best First'.
Re: Detect whether a writeable filehandle has closed?
by Perlbotics (Bishop) on Jan 13, 2021 at 18:51 UTC

    Writing to a closed pipe should trigger a SIGPIPE but it requires to write something to it first. A child that died for some reason should also trigger a SIGCHLD – perhaps you could detect that and take it into account prior to sending more data? I would write a wrapper that replaces print with a-priori checks (i.e. SIGCHLD sets a flag asynchonously) and post-checks (SIGPIPE)... at least under Linux. Don't know if theses assumptions are valid under Windows.

      Checking my copy of Stevens APUE my reading (and vague recollections of actual behavior) confirms this; you're not going to get SIGPIPE until you actually write to the fd the child's closed. The SIGCHLD should be the first sign you get.

      use IPC::Run qw( start pump ); use Log::Log4perl qw( :easy ); Log::Log4perl->easy_init($DEBUG); $SIG{$_} = eval qq{sub{INFO qq{Got $_}; 1}} for qw(CHLD PIPE); INFO q{Starting}; my $in=q{}; my $chld = start(["bash","-c","sleep 5"],q{<},\$in); ## This will get CLHD and sleep will exit early INFO q{Sleeping}; my $r=sleep(10); WARN( qq{$r, $!} ); ## This will trigger PIPE $in .= qq{FOO}; pump $chld; ## This won't be reached. INFO q{Done} __END__ 2021/01/13 15:07:42 Starting 2021/01/13 15:07:42 Sleeping 2021/01/13 15:07:47 Got CHLD 2021/01/13 15:07:47 5, Interrupted system call 2021/01/13 15:07:47 Got PIPE ack Broken pipe: write( 6, 'FOO' ) at ....

      The cake is a lie.
      The cake is a lie.
      The cake is a lie.

        With the implication that only the last write failed before the SIGPIPE so caching, or being able to regenerate, the written data may provide jdporter the recovery information he needs.

        Optimising for fewest key strokes only makes sense transmitting to Pluto or beyond

      Thanks! Those are definitely things to try. (I'm already catching SIGPIPE, but as I said I'm getting the signal "late".)

      But you bring to mind the idea that maybe I need to turn off buffering of the output!

Re: Detect whether a writeable filehandle has closed?
by stevieb (Canon) on Jan 13, 2021 at 17:43 UTC

    Hey jdporter, could you please provide the open that you're specifically using that ends up failing?

    I don't think I've ever used a pipe with open before, so before I try to test it out, an example would be great.

      open my $pipe, " | $other_program ";

      For testing purposes, the other program could be cat. In my case, it's amqsputc, a sample program included in the IBM MQ product. I'm on linux (OpenSUSE 12).

        Doing some searching it looks like they might support AMQP; you possibly could use something like Net::AMQP::RabbitMQ to connect to the broker directly rather than piping things through an external program. Probably at least give you other different problems to deal with . . .

        The cake is a lie.
        The cake is a lie.
        The cake is a lie.

Re: Detect whether a writeable filehandle has closed?
by haukex (Bishop) on Jan 13, 2021 at 17:45 UTC

    I'm not sure at the moment if this works for pipes, but you could try checking the return value of fileno.

      Are you suggesting that that might be able to detect a filehandle "gone bad"? My problem is not that the open fails, but that the filehandle goes bad (apparently being closed by the child) some time after opening.

        Are you suggesting that that might be able to detect a filehandle "gone bad"? My problem is not that the open fails, but that the filehandle goes bad (apparently being closed by the child) some time after opening.

        I see now what you mean, my assumption from the "Detect whether a writeable filehandle has closed?" was that you're looking at the filehandle on your end, but you're asking about the other end of the pipe. Sorry, fileno doesn't help with that, only if your filehandle closes (i.e. $pipe in your example here).

        You might want to look at IPC::Run, it has a pumpable method that seems more like what you're asking for. I've also used its pump_nb method inside an eval to detect when a child process has gone bad.

        Minor edits.

Re: Detect whether a writeable filehandle has closed?
by talexb (Canon) on Jan 13, 2021 at 17:33 UTC

    Maybe an END block in the child could signal that the filehandle is gone?

    Another approach would involve subclassing the filehandle .. ugh.

    Alex / talexb / Toronto

    Thanks PJ. We owe you so much. Groklaw -- RIP -- 2003 to 2013.

      Maybe an END block in the child could signal that the filehandle is gone

      The child is not perl, it's a compiled C program.

        OK, so perhaps using the _atexit call (or whatever the equivalent is in your RTL) in the C code. The general idea is for the code to catch that the child has exitted and signal the caller.

        Alex / talexb / Toronto

        Thanks PJ. We owe you so much. Groklaw -- RIP -- 2003 to 2013.

Re: Detect whether a writeable filehandle has closed?
by eyepopslikeamosquito (Bishop) on Jan 14, 2021 at 00:09 UTC
Re: Detect whether a writeable filehandle has closed?
by shmem (Chancellor) on Jan 14, 2021 at 14:11 UTC

    With unbuffered I/O, print returns undef when writing to a closed filehandle; for pipes, $! is then set to "Broken pipe". You don't get SIGPIPE after the child closed the pipe, but after an unsuccsessful write to that filehandle. I tried to use select (and IO::Select with $io->can_write()) to detect whether I can write to a filehandle before writing, to no avail. Both always report the filehandle to be apt for writing, even after receiving SIGPIPE. Only after closing the filehandle for writing in the parent (e.g. in the SIGPIPE handler) I get the expected result.

    On linux, that is (debian). So, no way to check whether the end point of the pipe is still connected before writing to it.

    perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'

      Thank you for that. I think I'd be ok if I got the SIGPIPE instantaneously, but as I said it appears that the signal has to "make its way through the system" before my handler code gets control, by which time I may have written any number of additional lines to the child. :-(

        As said, you get the SIPGPIPE immediately after the first unsuccessful write, so you may want to keep your amount of lines short for each write.

        If you have control of the child you could make it send a HUP or USR1 to its parent after closing the file handle.

        Update: Oh I see you haven't, a compiled C program... hm. A wrapper, maybe?

        perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'
Re: Detect whether a writeable filehandle has closed?
by ikegami (Pope) on Jan 15, 2021 at 02:50 UTC

    If you ignore SIGPIPE ($SIG{PIPE} = 'IGNORE';), the write will result in error EPIPE instead.

    But it sounds like you really want to detect the child exiting. Add a $SIG{CHLD} handler.

Re: Detect whether a writeable filehandle has closed?
by salva (Canon) on Jan 15, 2021 at 12:04 UTC
    Even if it were possible to check whether the pipe was still valid before writing to it, doing it would be a bad idea. You would be introducing a race condition as the child could die after you check the pipe but before the write operation is completed. In general, that's a bad programming practice.

    The right think to do is to catch the signal or the error (see ikegami's post above).

Re: Detect whether a writeable filehandle has closed?
by bliako (Prior) on Jan 14, 2021 at 12:36 UTC

    As others suggested the first thing to do would be to turn buffering off.

    Then simply check the pid is alive before every write.

    Which is essentially what you have suggested: to check if filehandle is open, but at a lower level, that of the process.

    A tiny caveat would be if your user does not have permission to check whether a pid is alive. I think that's unlikely in unix. For Windows you can always install one of the thousand viruses out there to hijack the system and bypass whatever permissions. Kids don't do this - someone else will do it for you (that's a joke)

    my $other_program = 'cat'; my $pid = open my $pipe, " | $other_program "; print $pipe "blahblah" if alive($pid);

    I thought something as simple as this: sub alive { return kill 0, $_[0] } would work but it doesn't in my Fedora-linux-latest. It always return true!

    If you are on a unix box, then it would be straightforward to check the pid using various methods (e.g. ps -o pid | grep -w $pid or in linux: via perl: -d "/proc/$pid". Additionally, relevant modules on CPAN do exist. It's unclear to me if they work for non-unix systems so I wont mention them here.

    bw, bliako

      Thank you. In fact, I had already tried the kill 0 technique, and it didn't work, just as you described.

      I'm going to try the /proc/$pid method. But I'm afraid that that might also yield a false negative if I haven't reaped the child yet. (ps shows "defunct".) (I suspect that that would also explain why kill 0 doesn't do what we expect either.)

        (ps shows "defunct".)

        You need waitpid to reap the zombie.

        perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'
Re: Detect whether a writeable filehandle has closed?
by eyepopslikeamosquito (Bishop) on Jan 15, 2021 at 22:00 UTC

    Thanks to ikegami and salva for their excellent advice.

    Having been traumatized writing signal handlers in the presence of non-reentrant system libraries, switching off the signal handler via $SIG{PIPE} = 'IGNORE' is what I would choose ... though I could not restrain myself from using strace on both the calling perl process and the recalcitrant IBM MQ amqsputc, in the hope of understanding what the hell is going on.

    BTW, since you share my passion for early Perl Monks history, you might enjoy the sage advice graciously contributed by faq_monk, one of PM's founding users, on Oct 8 1999 - more than two months before the Perl Monks web site opened to the general public!

    Update: from perlipc:

    The reason for not checking the return value from print() is because of pipe buffering; physical writes are delayed. That won't blow up until the close, and it will blow up with a SIGPIPE. To catch it, you could use this:

    $SIG{PIPE} = "IGNORE"; open(my $fh, "|-", "bogus") || die "can't fork: $!"; print $fh "bang\n"; close($fh) || die "can't close: status=$?";

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://11126853]
Approved by marto
Front-paged by marto
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (4)
As of 2021-04-19 23:07 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?