Event module and threads/fork segfault on Win32

[gurbo]

I've got a perl script running on Win32 using the Event module that uses Net::SSH::W32Perl to execute commands on remote machines. It has an io watcher looking for client's input (the commands to be executed on the remote hosts) and when it gets a command, creates another watcher of type timer (because the commands may be scheduled to be executed later). When the time comes, the watcher is invoked and it creates a thread to execute the remote commands (so the server doesn't freeze).
The thread executes ok but, when the server tries to join the thread, it segfaults. Same happens using fork but this is not a surprise because on Win32 its implemented using threads.
The problem seems to be related to spawning a thread while inside a watcher, because if I call fork outside a watcher, works ok and if I use fork inside any watcher (e.g. the io watcher) it segfaults.
Any help would be greatly appreciated.

[BrowserUk]

You are more likely to get help if you post your code, or a subset of it that demostrates the problem, than if you expect potential helpers to try and re-create your code from this sparse description of it.

[gurbo]

You're right. Here it is the quintessence of the problem:

#!/usr/bin/perl 
use strict;
use warnings;

use threads;


sub create_thread {
    warn "in create_thread() before async()\n";
    my $thr = async { warn "in thread\n"; };
    warn "in create_thread() before join()\n";
    sleep 2;
    $thr->join;
    warn "in create_thread() before return\n";
    return;
}


use Event;
my $timer = Event->timer( cb => \&create_thread, after => 2);

warn "starting event loop\n";
Event::loop;


When create_thread() returns, the interpreter segfaults.
The problem seems to be caused when join() is called, if you comment the call, the event loop ends and you get the warning:

Perl exited with active threads:
        0 running and unjoined
        1 finished and unjoined
        0 running and detached

[BrowserUk]

The next question is which versions of Perl, threads and Event are you using?

I just ran your script under AS811/5.8.6 with threads v1.05 and Event v1.09 and it completed without a segfault, and just a warning for a leaked scalar?

C:\test>619259.pl
starting event loop
in create_thread() before async()
in create_thread() before join()
in thread
in create_thread() before return
Scalars leaked: 1

C:\test>perl -v

This is perl, v5.8.6 built for MSWin32-x86-multi-thread
(with 3 registered patches, see perl -V for more detail)

Copyright 1987-2004, Larry Wall

Binary build 811 provided by ActiveState Corp. http://www.ActiveState.
+com
ActiveState is a division of Sophos.
Built Dec 13 2004 09:52:01

Perl may be copied only under the terms of either the Artistic License
+ or the
GNU General Public License, which may be found in the Perl 5 source ki
+t.

Complete documentation for Perl, including FAQ lists, should be found 
+on
this system using `man perl' or `perldoc perl'.  If you have access to
+ the
Internet, point your browser at http://www.perl.org/, the Perl Home Pa
+ge.


C:\test>perl -Mthreads -e"warn "$threads::VERSION"
1.05 at -e line 1.

C:\test>perl -MEvent -e"warn "$Event::VERSION"
1.09 at -e line 1.

[Joost]

I don't know much about win32 and Event, but:

a) is it possible that starting a new thread could cause the event loop to be run in multiple threads simultaneously? That would probably cause all kinds of problems.

b) if you can't start a thread within a watcher, can't you set a variable somewhere else (update: i mean in a shared/global scope) to signal that a thread should be started sometime (soon) after the watcher is done?

[chrism01]

Alternatively, is it really necessary to join the thr once it's done it's work?
Why not use thr->detach() instead?

Cheers
Chris

[gurbo]

The script needs to join() because some threads should run upon successful completion of others. So that's the need to know when they end and collect their exit statuses.

[zentara]

Hi, your code segfaults on linux too. Your create_thread looks funky to me, (maybe because you create the thread and join it all in one sub ?)

Since Event is an event loop system, I looked at it like I would a Tk or Gtk2 threaded program. In Tk, the thread must be created before any event-loop code( i.e. Tk statements ) are made, so the following seems to work for me.

#!/usr/bin/perl 
use strict;
use warnings;
use threads;

my $thr = async {  
             for(1..10){
               warn "in thread $_\n";
               sleep 1;
              }
           };

sub create_thread {
    warn "in create_thread() before async()\n";
#    my $thr = async { warn "in thread\n"; };
    warn "in create_thread() before join()\n";
    sleep 2;
    $thr->join;
    warn "in create_thread() before return\n";
    return;
}

use Event;
my $timer = Event->timer( cb => \&create_thread, after => 2);

warn "starting event loop\n";
Event::loop;

Gtk2 has a thread-safety mechanism, that does allow you to create threads after event-loop code has been written. So you might want to try Gtk2 and it's GLib. However, even some Gtk2 experts say it is still better to create your threads first, before any Gtk2 code is written. See Roll your own Event-loop

[gurbo]

I create the thread and join it inside the same watcher just to make things simple. In real life, there is a watcher that starts them on demand and there is another that joins them to collect their exit statuses.
That's why I cannot start the thread at the start of the script. They are created when a client makes a request.

Thanks for the link to Glib, I'll give a try, but I cannot found Glib nor Gtk2 in ActiveState Perl package manager. Searching in Google I found gtk2-perl Win32 Binary Home. Is this the right place?

[zentara]

I don't use Win32, but try Install GTK2 or gtk2-win32 support

Glib is the base library for Gtk2. It does not need a gui and is developed to be thread-safe( as demonstrated by my previous example). I would be willing to bet that Event.pm is not thread-safe. The author talks alot about threads in the Event.pm perldoc, but never mentions the thread-safety of Event.pm

So I'm willing to bet it acts like Tk, and you have to start the threads before any Tk code is called.... I generally use a "sleeping thread" where I have a thread code block controlled by a $go shared variable. That way you can use Event, by starting the thread before any Event code is written, then in your create_thread callback, just set $go to 1 to get the thread running. If you need help on how to do that, just use the SuperSearch for "Tk threads" and you will find many examples of a sleeping thread.

This is from the Perl/Gtk2 maillist:

From: Gabor Szabo <[e-mail]>

For ActiveState users

* I assume you already have ActiveState Perl installed on your system.
If not, download and install ActiveState Perl from
http://activestate.com/Products/ActivePerl/

* Next you need to install the GTK+ 2.x.x runtime environment. The
latest version can be downloaded from
http://gimp-win.sourceforge.net/stable.html

* Ensure that the <INSTALL_PATH>\GTK\2.0\bin directory is in your
PATH after this installation.

* Next you need to install the Win32 binary packages for gtk2-perl.
Download and install the latest version of Gtk2.ppd and Glib.ppd from
http://gtk2-perl.sourceforge.net/win32/ppm/
(Another way to install these ppds is as follows
C:\Temp> ppm install http://gtk2-perl.sourceforge.net/win32/ppm/Gtk2.p
+pd
C:\Temp> ppm install http://gtk2-perl.sourceforge.net/win32/ppm/Glib.p
+pd
)

Your system should be ready to roll !

[zentara]

Hi, this works on linux, where I start a thread after 4 seconds, then stop it after 8. It uses Glib.

#!/usr/bin/perl
use warnings;
use strict;
use Glib;
use Glib qw/TRUE FALSE/;
use threads;
use threads::shared;

my $count = 0;
my $thread;
my $die:shared = 0;

my $main_loop = Glib::MainLoop->new;
#test timer       
my $timer  = Glib::Timeout->add (1000, \&timer_callback, undef, 1 );

#timer to start thread after 4 seconds
my $timer1  = Glib::Timeout->add (4000, \&timer1_callback, undef, 1 );

#timer to stop thread after 8 seconds
my $timer2  = Glib::Timeout->add (8000, \&timer2_callback, undef, 1 );


$main_loop->run;

sub timer_callback{
     $count++;
     print "$count\n";
     return 1;
}

sub timer1_callback{
     $thread = threads->new(\&work);
     return 0;  #run once only
}

sub timer2_callback{
     $die = 1;
     $thread->join;
     return 0;  #run once only
}


sub work{
    
    $|++; 
    
    while(1){
         foreach my $num (1..1000){
                
                if($die){return}
                     
                 Glib::Idle->add(
               sub{
                print "\t\t$num\n";
            return FALSE;
           });
                
            select(undef,undef,undef, .1);

         }
       
    }
}

__END__ 

#my $timer  = Glib::Timeout->add ($interval, $callback, $data=undef, $
+pri-
#      ority=G_PRIORITY_DEFAULT)

#    * $interval (integer) number of milliseconds
#    * $callback (subroutine)
#    * $data (scalar)
#    * $priority (integer)

#    Run $callback every $interval milliseconds until $callback return
+s
#    false.  Returns a source id which may be used with
#   "Glib::Source->remove".  Note that a mainloop must be active for
#    the timeout to execute.