Re^13: Forks, Pipes and Exec (file descriptors)

[diabelek]

It's a command line IO tool that we use that isn't ours so I changed the name. I just place it in c:\windows since its in the %PATH% and easier to access

Basically you have the standard output of any IO tool such as the options set, IO/s, errors, warnings, status, etc. If you have ever used iozone or dtstart in Linux, the output would be similar.

The way the script is right now, the tool doesn't matter since I'm still trying to get the worker threads going that will handle gathering the output from each instance of wintool.exe. Unless there's a better way of implementing it in perl, I'm stuck on why the thread won't start.

update:

I slowed the script down so I could see whats going on and where its hanging and apparently it hangs completely in the StartIO function when I try to read back the results from the socket. If I change the <$rfh> to this:

   $rfh_select = new IO::Select( $rfh ) or die "Error setting up IO::S
+elect: $!\n";
   while( !(@ready = $rfh_select->can_read( .1 )) ) {
      #can_read hangs for .1 seconds so wait for it
      # need to add some code to stop the loop if we wait to long
   }
   sysread( $rfh, $results, 9999999 );

It will run more or less like it should. Still a few bugs to work out but at least the thread isn't hanging when I ask it to start.

So the million dollar question that I'm asking is what would a <$handle> in thread A stop a threads->new() from starting thread C & D from thread B?

[BrowserUk]

so I changed the name.

Hm. An unfortunate but coincidental 'random' choice of name? (Google to understand what I mean.)

So the million dollar question that I'm asking is what would a <$handle> in thread A stop a threads->new() from starting thread C & D from thread B?

To be able to answer that question with any certainty, I'd have to be able to run code that demonstrates the problem with my debugger and other tools. And a part of that would be using the actual executable to see (for example) whether it is buffering its output.

I'd also have to see what is going on inside the thread that is 'failing to start'. And clearer definition of 'failing to start' wouldn't go amiss.

However, let me speculate. Does the thread that fails to start attempt to either read or write to the same socket ($handle)?

Rational: using readline (including via the diamond (<>) operator), will block, even on a socket set non-blocking, until a newline is read. That's the basic definition of readline().

And, you cannot (under win32 at least), both read from and write to the same socket concurrently on different threads. The OS will serialise concurrent access attempts.

And if the external process is buffering it's output, then even if every line it writes is always terminated with a newline, the effect of buffering means that when the buffer size, (probably 4096 bytes), is eventually reached, several lines--usually including an incomplete, unterminated last line--will be flushed as a single output.

If the receiving end of the pipe is doing readlines, then it will block waiting for that incomplete line to be completed. But, the external process may take a while to (or indeed never), fill another buffer load, and so the receiving end just sits and waits. If the other threads you are starting then attempt to write to the socket, they will block until the reading thread completes it's readline. Which may never happen.

When you set sockets non-blocking, you should avoid buffered IO. Instead use either sysread or recv and syswrite or send.

That's all speculation (given the absence of good information or code to test), and may only be applicable to win32