Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

threads->create() blocks on Windows

by photron (Novice)
on Mar 17, 2014 at 17:26 UTC ( [id://1078634]=perlquestion: print w/replies, xml ) Need Help??

photron has asked for the wisdom of the Perl Monks concerning the following question:

Hi!

I have a problem with using threads and sockets on Windows: threads->create() blocks. The following code is a simplified part of a bigger project in which I found this problem:

use strict;
use warnings;
use threads;
use IO::Socket::INET;

my $s = IO::Socket::INET->new(PeerAddr => 'www.google.com',
                              PeerPort => 80,
                              Proto => 'tcp') or die "error: $@";

print "create t1\n";

my $t1 = threads->create(sub {
    my ($r) = @_;
    my $data = '';

    print "in t1\n";

    while (1) {
        $r->recv($data, 64);
        #print ">>> $data\n";
    }
}, $s);

# give t1 a moment to execute and to block in the recv call
print "sleep 3s\n";
sleep(3);
print "sleep done\n";

print "create t2\n";

my $t2 = threads->create(sub {
    # problem: this is never printed
    print "in t2\n";

    $s->send("GET / HTTP/1.1\r\n\r\n");
}, 0);

# problem: this is never printed, as threads->create never returns
print "done\n";

$t1->join();
$t2->join();

[download]

The code opens a socket to www.google.com. Google is just used for demonstration purpose. The actual project connects to a special server using a simple request/response protocol. Then thread t1 is created to receive all incoming data. After that I injected a sleep(3) to ensure that t1 is blocking in the recv() call before going on. This is important, the problem of threads->create() hanging only occurs if t1 is blocking in recv(). Finally t2 is created and should send a request. But this never happens on Windows, because threads->create() for t2 never returns.

This problem only occurs on Windows (tested with Strawberry and Active Perl 5.18.2). It doesn't occur on Linux, there this just works as expected.

Am I doing something wrong here? Is there some general problem with this approach that could explain the problem? Or is this a bug in perl itself?

Replies are listed 'Best First'.
Re: threads->create() blocks on Windows
by BrowserUk (Patriarch) on Mar 17, 2014 at 20:21 UTC

    Your first thread puts the socket into a blocking read state.

    When you attempt to start the second thread, Perl tries to clone $s for that thread, because it closes over $s.

    But, the underlying system handle for the socket is locked (by the OS) because the socket is in a blocking read; thus the attempt to clone the handle is blocked until the read completes; which it never will.

    As the clone is blocked, the second thread is never started; so no GET is issued, so no reply will be received.

    The solution (maybe, depending upon the rest of your application,) it to ensure the get occurs before the read state is entered.

    Something like this:

    use strict;
use warnings;
use threads;
use IO::Socket::INET;

$|++;

my $s = IO::Socket::INET->new(PeerAddr => 'www.google.com',
                              PeerPort => 80,
                              Proto => 'tcp') or die "error: $@";

print "create t2\n";

my $t2 = threads->create(sub {
    # problem: this is never printed
    print "in t2\n";

    $s->send("GET / HTTP/1.1\r\n\r\n");
}, 0 );

# give t1 a moment to execute and to block in the recv call
print "sleep 1s\n";
sleep(1);
print "sleep done\n";

print "create t1\n";

my $t1 = threads->create(sub {
    my ($r) = @_;
    my $data = '';

    print "in t1\n";

    while( 1 ) {
        $r->recv($data, 64);
        print ">>> $data";
    }
}, $s);



# problem: this is never printed, as threads->create never returns
print "done\n";

$t1->join();
$t2->join();

    [download]

    Outputs:

    C:\test>junk41
create t2
in t2
sleep 1s
sleep done
create t1
done
in t1
>>> HTTP/1.1 302 Found
Cache-Control: private
Content-Type: text/h>>> tml; charset=UTF-8
Location: http://www.google.co.uk/?gfe_rd=cr>>> &ei=-lgnU7OuDdT88QPQ84
+CwAw
Content-Length: 261
Date: Mon, 17 M>>> ar 2014 20:20:10 GMT
Server: GFE/2.0
Alternate-Protocol: 80:qu>>> ic

<HTML><HEAD><meta http-equiv="content-type" content="text/>>> html;cha
+rset=utf-8">
<TITLE>302 Moved</TITLE></HEAD><BODY>
<H1>3>>> 02 Moved</H1>
The document has moved
<A HREF="http://www.google.>>> co.uk/?gfe_rd=cr&amp;ei=-lgnU7OuDdT88QP
+Q84CwAw">here</A>.
</BOD>>> Y></HTML>
Terminating on signal SIGINT(2)

    [download]
    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
[select]
      Thanks for the detailed explanation. This seems exactly to be the problem. I cannot create a new thread while another thread does a blocking receive on a global socket. So I'll have to avoid this situation.
[reply]
      But, the underlying system handle for the socket is locked (by the OS) because the socket is in a blocking read; thus the attempt to clone the handle is blocked until the read completes; which it never will.

      Well, this explanation seemed reasonable to me and it seems to indicate that the problem is in Windows itself and that Perl just happens to work in a way that collides with some Windows behavior here if I write my script in a certain way.

      I doubt this now! I had tested this script with Strawberry Perl and Active State Perl and both show the same behavior. But now I tested this with Cygwin's Perl package and it just works. So is this actually a bug in Strawberry and Active State Perl, or is Cygwin just applying some magic here to make this work?

[reply]
        So is this actually a bug in Strawberry and Active State Perl, or is Cygwin just applying some magic here to make this work?

        Both AS Perl and Strawberry Perl use the MS C-runtime, which is where the internal locking is done. It's not an error, it's a design choice.

        Cygwin on the other hand implements a strictly POSIX-complient C-runtime which makes other choices.

        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
Re: threads->create() blocks on Windows
by bulk88 (Priest) on Mar 18, 2014 at 02:50 UTC
    I see you earlier asked in #win32. In your script 
        use strict;
    use warnings;
    use threads;
    use IO::Socket::INET;
     
    my $s = IO::Socket::INET->new(PeerAddr => 'localhost',
                                  PeerPort => 80,
                                  Proto => 'tcp') or die "error: $@";
     
    print "create t1\n";
     
    my $t1 = threads->create(sub {
            my ($r) = @_;
            my $data = '';
     
            print "in t1\n";
     
            # this intentionally blocks. if this line is removed the p
+roblem vanishes
            $r->recv($data, 64);
     
            print "recv done\n";
    }, $s);
     
    print "create t2\n";
     
    my $t2 = threads->create(sub {
            print "in t2\n"; # problem: this is never printed
    }, 0);
     
    print "done\n"; # problem: this is never printed, as threads->crea
+te never returns
     
    $t1->join();
    $t2->join();

    [download]


    Your code freezes like this.

    Parent thread. 
         ntdll.dll!_KiFastSystemCallRet@0()     
     ntdll.dll!_NtQueryVolumeInformationFile@20()  + 0xc    
     kernel32.dll!_GetFileType@4()  + 0x72    
>    msvcr71.dll!_fstati64(int fildes=3, _stati64 * buf=0x0012fab4)  L
+ine 97 + 0x8    C
     perl519.dll!PerlLIOFileStat(IPerlLIO * piPerl=0x00cdc134, int han
+dle=3, _stati64 * buffer=0x0012fab4)  Line 971 + 0xe    C
     perl519.dll!PerlIOUnix_setfd(interpreter * my_perl=0x00000400, _P
+erlIO * * f=0x0012f9cc, int fd=1029, int imode=0)  Line 2571 + 0x15  
+  C
     perl519.dll!PerlIOUnix_dup(interpreter * my_perl=0x00b4b014, _Per
+lIO * * f=0x00b4bac4, _PerlIO * * o=0x00905b24, clone_params * param=
+0x0012fb9c, int flags=1)  Line 2696 + 0xf    C
     perl519.dll!PerlIOBase_dup(interpreter * my_perl=0x00b4b014, _Per
+lIO * * f=0x00b4bac4, _PerlIO * * o=0x008f5574, clone_params * param=
+0x0012fb9c, int flags=1)  Line 2221    C
     perl519.dll!PerlIO_fdupopen(interpreter * my_perl=0x0012f9cc, _Pe
+rlIO * * f=0x00070023, clone_params * param=0x0012fb9c, int flags=1) 
+ Line 503 + 0x13    C
     perl519.dll!Perl_fp_dup(interpreter * my_perl=0x00b4b014, _PerlIO
+ * * const fp=0x008f5574, const char type=0, clone_params * const par
+am=0x0012fb9c)  Line 11825 + 0xb    C
     perl519.dll!PerlIO_clone(interpreter * my_perl=0x0000006c, interp
+reter * proto=0x00000400, clone_params * param=0x0012fb9c)  Line 617 
++ 0xc    C
     perl519.dll!perl_clone_using(interpreter * proto_perl=0x0036471c,
+ unsigned long flags=6, IPerlMem * ipM=0x00000000, IPerlMem * ipMS=0x
+00cdc028, IPerlMem * ipMP=0x00cdc044, IPerlEnv * ipE=0x00cdc060, IPer
+lStdIO * ipStd=0x00cdc098, IPerlLIO * ipLIO=0x00cdc134, IPerlDir * ip
+D=0x00cdc19c, IPerlSock * ipS=0x00cdc1c8, IPerlProc * ipP=0x00cdc278)
+  Line 13649    C
     perl519.dll!perl_clone_host(interpreter * proto_perl=0x0036471c, 
+unsigned long flags=6)  Line 360    C
     threads.dll!S_ithread_create(interpreter * parent_perl=0x00000400
+, sv * init_function=0x008fcc14, long stack_size=0, int gimme=2, int 
+exit_opt=0, int params_start=3, int num_params=1)  Line 794    C
     threads.dll!XS_threads_create(interpreter * my_perl=0x00000002, c
+v * cv=0x0096de44)  Line 1100 + 0x30    C
     perl519.dll!Perl_pp_entersub(interpreter * my_perl=0x00000000)  L
+ine 2794    C
     perl519.dll!Perl_runops_standard(interpreter * my_perl=0x0036471c
+)  Line 42 + 0x4    C
     perl519.dll!S_run_body(interpreter * my_perl=0x0000006c, long old
+scope=1)  Line 2449 + 0xa    C
     perl519.dll!perl_run(interpreter * my_perl=0x0036471c)  Line 2365
+ + 0x8    C
     perl519.dll!RunPerl(int argc=2, char * * argv=0x01362478, char * 
+* env=0x00362d80)  Line 259 + 0x6    C
     perl.exe!mainCRTStartup()  Line 398 + 0xe    C
     kernel32.dll!_BaseProcessStart@4()  + 0x23

    [download]
    Child thread 
         ntdll.dll!_KiFastSystemCallRet@0()     
     ntdll.dll!_ZwDeviceIoControlFile@40()  + 0xc    
     mswsock.dll!_WSPRecv@36()  + 0xd1    
     mswsock.dll!_WSPRecvFrom@44()  + 0xe099    
     ws2_32.dll!_recvfrom@24()  + 0x87    
>    perl519.dll!win32_recvfrom(unsigned int s=3, char * buf=0x00b376f
+4, int len=64, int flags=0, sockaddr * from=0x0117fa10, int * fromlen
+=0x0117fe50)  Line 481 + 0x34    C
     perl519.dll!PerlSockRecvfrom(IPerlSock * piPerl=0x00aad5e0, unsig
+ned int s=3, char * buffer=0x00b376f4, int len=64, int flags=0, socka
+ddr * from=0x0117fa10, int * fromlen=0x0117fe50)  Line 1389 + 0x17   
+ C
     perl519.dll!Perl_pp_sysread(interpreter * my_perl=0x00000400)  Li
+ne 1699    C
     perl519.dll!Perl_runops_standard(interpreter * my_perl=0x0090865c
+)  Line 42 + 0x4    C
     perl519.dll!Perl_call_sv(interpreter * my_perl=0x00158018, sv * s
+v=0x003b92b0, volatile long flags=18348252)  Line 2764 + 0xc    C
     threads.dll!S_ithread_run(void * arg=0x00905c24)  Line 520 + 0x14
+    C
     kernel32.dll!_BaseThreadStart@8()  + 0x37

    [download]
    Using Process Explorer, which can give a kernel side stack dump (but its HIGHLY INACCURATE since I've never figured out how to make Proc Exp use symbol files). 
    ntoskrnl.exe!ZwAlertThread+0x28
ntoskrnl.exe!SeAssignSecurityEx+0x13f
ntoskrnl.exe!MmGrowKernelStack+0x61a
ntoskrnl.exe!IoPageRead+0xaf0
ntoskrnl.exe!MmGrowKernelStack+0x659
ntoskrnl.exe!RtlGenerate8dot3Name+0x1ca
ntoskrnl.exe!IoCheckFunctionAccess+0xf841
ntoskrnl.exe!ZwSetSystemInformation+0x23
ntdll.dll!KiFastSystemCallRet
MSVCR71.dll!fstati64+0x63
perl519.dll!win32_stderr+0x302
perl519.dll!PerlIO_teardown+0x3ea
perl519.dll!PerlIOBase_dup+0x33
perl519.dll!PerlIO_allocate+0x8b
perl519.dll!Perl_fp_dup+0x32
perl519.dll!perl_clone_using+0x74f
perl519.dll!perl_clone_host+0x6f
threads.dll+0x2647

    [download]
    IDK why a GetFileType on a socket while its doing a recv in another thread blocks the GetFileType. If I kill the child OS thread doing the recv (and therefore the ithread with it), the GetFileType/NtQueryVolumeInformationFile unblocks and execution in the parent thread continues.
[reply]
[d/l]
[select]
      Yes, I asked this on #win32 with a slightly different example. The problem is the same in both. As you showed in the traces, the blocking recv() blocks the duplication of the socket for the new thread t2. So I have to avoid this order of execution. Thanks for the hints, hopefully I can modify the program in the correct way to make it work :)
[reply]
Re: threads->create() blocks on Windows
by Anonymous Monk on Mar 17, 2014 at 18:22 UTC

    Set your stdout to autoflush, and try again.
    $| = 1;

[reply]
[d/l]

      I tried this but it didn't make a difference.

[reply]

Back to Seekers of Perl Wisdom

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?

www.com | www.net | www.org
Node Status?
node history
Node Type: perlquestion [id://1078634]
Front-paged by Arunbear
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? • Last hour • Other CB clients
Other Users?
Others rifling through the Monastery: (4)
As of 2024-04-23 23:15 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found