Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine

Re^4: Forks, Pipes and Exec (file descriptors)

by BrowserUk (Patriarch)
on Nov 06, 2008 at 19:17 UTC ( [id://722061]=note: print w/replies, xml ) Need Help??

in reply to Re^3: Forks, Pipes and Exec (file descriptors)
in thread Forks, Pipes and Exec

...even though they are separate processes.

On win32, fork is not a separate process. It is a thread of the same process masquarading as a fork.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
  • Comment on Re^4: Forks, Pipes and Exec (file descriptors)

Replies are listed 'Best First'.
Re^5: Forks, Pipes and Exec (file descriptors)
by diabelek (Beadle) on Nov 06, 2008 at 19:44 UTC

    If it is a separate thread, is there any way to have STDOUT for thread A be different from thread B?

    From the activestate documentation, the STDOUT filehandle should have been duped for the child thread but I don't believe that's what I'm seeing. I can make a change in the parent to STDOUT that affects the child's STDOUT. Can anyone explain what I'm missing?

    active state documentation
    "Open filehandles
    Any filehandles open at the time of the fork() will be dup()-ed. Thus, the files can be closed independently in the parent and child, but beware that the dup()-ed handles will still share the same seek pointer. Changing the seek position in the parent will change it in the child and vice-versa. One can avoid this by opening files that need distinct seek pointers separately in the child."

      I can make a change in the parent to STDOUT that affects the child's STDOUT.

      It depends what change you are making and how you are perceiving that the the child has been affected.

      In general, I do not consider Perl's Win32 fork emulation worth the effort of bothering with. There are simply too many differences between the platforms and holes in the emulation, that make *nix techniques fail to work. A few examples:

      • In your snippet above, you create a pipe and set it up as stdout and then you attempt to set it non-blocking, presumably with the intent of use select upon it.

        But win32 anonymous pipes cannot be set non-blocking and so cannot be used in conjunction with select.

        Note:Win32 pipes can be used in a non-blocking fashion--using peeking (polling) or asynchronous IO or overlapped IO, but none of these fit well with the select mode of operations. This is not a limitation, but rather a difference in design philosophy that is hard to reconcile in a portable fashion.

      • If you do a fork followed by exec on *nix, then the pid returned to the 'parent' is process handle to the newly started process.

        If you do the same thing using the win32 emulation, the 'pid' is actually a disguised thread handle to a thread, and the process you "exec'd" is actually started as an entirely new process (using what is effectively the same as system), with an entirely different pid to that returned by fork--and to which you can never get access.

        Consequently, anything the parent does with that pseudo-pid only affects the thread--not the "exec'd" process. Eg. killing the pseudo-pid (or attempting send almost any signal) will terminate the thread, but leave the process running.

        Again, it is perfectly possible to set up bi-directional communications between a parent and child process under win32 at the C-level, but far harder to see how to make this available to Perl programs ina platform agnostic way.

      • Signals are not implemented (by the system) on win32, and only a handful (2,3,15,21) are emulated in a way that can be trapped.

        That means you will never receive a SIGCHLD in the parent. Whilst wait and waitpid are emulated, it is done by having the child thread block in a system wait on the child process, which makes for very different and confusing semantics.

      Those are just a few of the limitations that come to mind. I've encountered several more anomalies that don't. IMO these difference make trying to code portable programs using fork unworkable if win32 is a target platform.

      If you application calls for single direction piping of data to or from the child, then using the forked-open is effective and far more portable. If you bi-directional communications between parent and child, then threads can provide an effective solution that is again, far more portable than fork.

      Your question to date simply asks about forking and pipes ,without giving any hint as to the actual application, so it is impossible to suggest alternatives to that approach.

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

        I'll answer some of the points you brought up first so you can point out any flaws in my thinking:

        1. I'm using the non-blocking because I have a thread that executes a requested command that should only take a few milliseconds. It then reads the output pipes from all my external applications and stores whatever it read into a buffer. If there isn't anything on the pipe that doesn't matter, I just continue and service the next request that may/may not be waiting. Select is never used so I didn't see this as an issue.
        2. The only signal I need is to tell the process to die. I've been using SigKill since I just want the forked executable to die. This seems to work in Windows and Linux.
        3. Thank you for the info on open. I thought the open( fh, "cmd |") ran in the foreground. I've switched to using that and it has cut out quite a bit of code. It also seems to work even better for killing the process. I'm still using sigkill since I don't know of a better way to do it in perl. Suggestions?
        4. The goal of the module, briefly addressed in the first point, is to provide an instance of the module that can start another process/thread/etc. The two processes will communicate with each other in a command/result xml format (all commands come from the instantiated module and are sent to the child process). The child process is the responsible for servicing those requests which includes starting up additional cli programs and buffering their output. The buffer will be regularly checked for errors and when the parent process requests the errors, they are passed back up. I hope that explains things.

        From what I've read POE might have a been something to look at but I need this done sooner than later and I already have this module working in Linux... Windows is always the problem (jab). Plus I would like to get a better understanding of the inner workings of perl

        The last problem I seem to be fighting is that when I run the code to set a pipe as nonblocking, it isn't really nonblocking. I seem to get a chunk of data every 15 seconds or so on Win2k3 using this code:

        use IO::Handle; print "before the pipe\n"; sub getpipe { my $hash2 = shift; $hash2->{pid} = open( $hash2->{stdout}, "c:\\perl\\bin\\perl.exe ye |") or die; $hash2->{buffer} = ""; $hash2->{stdout}->blocking(0); sleep( 3 ); return $hash2->{pid}; } my $hash = {}; getpipe($hash); my $fh = $hash->{stdout}; print( "pipe created\n" ); while( <$fh> ) { print "got from pipe: $_"; sleep( 1 ); } print( "done\n" );

        while( 1 ) { printf "%s yes\n", time(); sleep( 1 ); }

        sample output:

        got from pipe: 1226094472 yes 1226094473 yes 1226094474 yes 1226094475 yes 1226094476 yes 1226094477 yes 1226094478 yes 1226094479 yes 1226094480 yes 1226094481 yes 1226094482 yes 1226094483 yes 1226094484 yes 1226094485 yes 1226094486 yes 1226094487 yes

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://722061]
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others cooling their heels in the Monastery: (1)
As of 2024-07-19 02:26 GMT
Find Nodes?
    Voting Booth?

    No recent polls found

    erzuuli‥ 🛈The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.