Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

open2() in Windows

by bhaveshbp (Initiate)
on Jul 22, 2007 at 03:01 UTC ( #628063=perlquestion: print w/ replies, xml ) Need Help??
bhaveshbp has asked for the wisdom of the Perl Monks concerning the following question:

Perl experts, I am in the process or porting a program written to work in Unix to run in Windows. At one point in the program it uses open2() to communicate with another program. The problem is in Windows the program hangs at random times after writing to the write pipe. By random times I mean, the program writes to the write pipe inside of a for loop and the program hangs at different iterations of the loop everytime after writing to the write pipe. The program just hangs and I get no error message and I have to manually kill the program. I dont see this issue when I run the program in Unix. I was thinking its somekind of Windows buffer issue because the loop executes thousands of times. Anyone got any ideas?

Comment on open2() in Windows
Re: open2() in Windows
by almut (Canon) on Jul 22, 2007 at 11:13 UTC

    Could you show us some reasonably short piece of code that reproduces the problem? Also, it would help to know what the program being called is doing, in particular the specifics of how it's handling reads and writes to the pipe. Smooth bidirectional communication depends as much on the behaviour of the process at the other end of the pipe, as it does depend on the side calling the open2.

    For example, it could be some deadlock situation arising due to different buffer sizes involved, different timings, etc. (I.e. both sides are waiting for the other side to make the next step — which means that, without external intervention, they'd remain in that unfortunate state forever...). However, that's just a shot in the dark... I don't think there's any glaringly obvious explanation for why things are no longer working, so more details might help to generate other hypotheses.

      I cant really post any code because its work related and cant show any code. But ill try and recreate what happens abstractly.
      open2(read,write,program); for (executes about 6000 times) { print write $cmd; #This works fine and dandy until about usually 3500 iter +ations into the loop, and its the same command everytime. Sometimes +I even get to 5000 iterations into the loop before the program hangs. }
      Whats weird is if I open up the program manually and execute the commands that my script does by hand everything works fine. This is why I thought it was some kind of Windows buffer error.
Re: open2() in Windows
by BrowserUk (Pope) on Jul 23, 2007 at 08:35 UTC

    You are connecting to both stdin & stdout, but never processing stdout. Consequently, if the child process produces any output, it will eventually fill the pipe that you are not reading and the OS will block the child from writing anymore until you do something in the parent to reduce the content of the pipe.

    The number of iterations it will run will depend upon the amount of output produced by the child. Ie. How quickly it fills the pipe buffer. Eg. In the following which just echos the 'some command's it reads, the iteration count gets to 83 before it blocks.

    #! perl -slw use strict; use IPC::Open2; $|++; my $pid = open2 \*READ, \*WRITE, 'perl -wne "$|++; chomp; print"'; my $read = ''; for ( 1 .. 1e4 ) { Win32::Sleep( 10 ); printf "\r$_\tGot'$read'\t"; print WRITE 'some command'; } __END__ c:\test>junk Name "main::READ" used only once: possible typo at c:\test\junk.pl lin +e 7. 83 Got'' Terminating on signal SIGINT(2)

    83 * 13 = 1079, suggesting roughly a 1k buffer.

    Reduce the output to a single char:

    #! perl -slw use strict; use IPC::Open2; $|++; my $pid = open2 \*READ, \*WRITE, 'perl -wne "$|++; chomp; print 1"'; my $read = ''; for ( 1 .. 1e4 ) { Win32::Sleep( 10 ); printf "\r$_\tGot'$read'\t"; print WRITE 'some command'; } __END__ c:\test>junk Name "main::READ" used only once: possible typo at c:\test\junk.pl lin +e 7. 553 Got'' Terminating on signal SIGINT(2)

    This time the iteration count gets to 553, which means that the size of the buffer is a little more complicated.

    But if you process the output from the child within the parent's loop:

    #! perl -slw use strict; use IPC::Open2; $|++; my $pid = open2 \*READ, \*WRITE, 'perl -wne "$|++; chomp; print 1"'; my $read = ''; for ( 1 .. 1e6 ) { # Win32::Sleep( 10 ); printf "\r$_\tGot'$read'\t"; print WRITE 'some command'; sysread( READ, $read, 1024 ); } __END__ c:\test>junk 1000000 Got'1'

    Now the loop happily runs for as long as you care to leave it running. The OS is simply protecting you from yourself, by preventing you from consuming all your memory by filling a pipe with output that you are never processing.

    Note: I used the non-blocking sysread to process the output as the child process is never producing newlines, so <READ> or read would block.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      Sorry I forgot to mention in my previous post that inside the parent loop it is processing the output from the child process. But not with the sysread() function. We are doing it like this:
      print WRITE $cmd while (<READ>) { if (/blah/) { do something } else { do something else } }
        But if I do it this way:
        $|++; open2(READ,WRITE,program); for (1..6000) { print WRITE $cmd; while (<READ>) { if (/blah/) { do something; } else { do another thing; } } }
        This should flush the pipes everytime right because im setting $| to something greater than zero? Thus, the pipes should never be full?

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://628063]
Approved by planetscape
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (9)
As of 2014-10-23 01:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    For retirement, I am banking on:










    Results (123 votes), past polls