5005threads -> ithreads (porting)

[castaway]

Since I attempted several times to convert a script of mine from 5005threads to ithreads, and then to make it work optionally with both, I thought I'd share my experiences (and see if anyone has any improvements/ideas on my technique.)

It's a largish script which has two threads. The main thread spawns off the second thread, which is essentially running a telnet connection, and then starts to loop waiting for input from the user. (Yup, its a telnet/mud client..). I had fun getting it to work originally, as I was not using any Thread::Queues (and now realise I should have been, would have saved me lots of hassle); so there were deadlock problems when the telnet thread wanted to write to the screen (via a Term::Screen object), and when the input thread needed to write to the telnet connection. (Anyone seeing this simple description would probably think its tailor made for Queues, but I was just learning thread programming at the time, and didnt see this simplicity, there appeared to be a lot more interfaces between the two.)

Anyway:
The first few attempts failed completely, mostly because sharing objects between ithreads is not an option (it works fine with 5005threads), which lead me to believe it was impossible to port, or at least extremely difficult. ie. I couldn't share the Term::Screen object, or the telnet object. My first thought was that maybe I could work with 2 Term::Screen objects (after all, theres only one screen, and I was setting the cursor position each time before printing..) - which worked. But didnt solve my problem with the telnet object.. At this point I discovered Thread::Queue (yup, why didnt I read perlthrtut earlier? - I had, but hadn't understood how easy they are..) After setting up a Queue whereby the input thread stuffs any text meant to be sent over the telnet connection into a queue, and the telnet thread dequeues these and sends them, I realised that actually I didnt need to share any objects at all. (Note: perlthrtut doesn't mention it, but Queues also have a non-blocking 'dequeue' method, 'dequeue_nb', which is more useful than the blocking one in my opinion, or you can use 'pending' to see if anything is actually in the Queue.) I then created a second queue to control the traffic flowing in the other direction (screen output from the telnet thread), and lo, everything worked..

This was using 'use threads; use threads::shared' - ie just testing on perl 5.8.0 with ithreads. My goal was to get it to work with both, depending on which was available. To check for threads support and version, just a

if($Config{5005threads}) { .. }
elsif($Config{ithreads}) { .. }

is needed. So I changed my 'use's to 'require's, and lo, everything continued to work..

Until I realised I did need to share one variable, $CONNECT, which is set by either thread to indicate that the connection is to be closed (either because it was closed by the server, or by the user). I tried to share this variable, and managed to seg fault perl - the segfault was on the 'share($CONNECT);' line, without that, everything ran fine.. (Fun) liz then proceeded to tell me that 'require threads::shared' isn't an option, as it needs to be done at compile time. So I tried to stick my $Config tests and requires in a BEGIN block, to no avail (still segfaulted). Then I swapped 'require' for 'use', which seems to ignore the BEGIN block, and attempt to load both Thread and threads, and produces a list of 'sub redefined's.

A temporary solution was to use an extra Thread::Queue object in which a '1' is enqueued if either thread wishes to indicate a close. Strangely enough, Thread::Queue also just works using shared arrays and condition signals, but continues to work, even without threads::shared. (Note: Aha, Thread::Queue 'use's threads::shared itself! Hmm..)

*some testing and 5.8.1 later* This segfaults (5.8.0):

my $CONNECT = 0;
..
require Thread::Queue;
threads::shared::share($CONNECT);

Also if the $CONNECT is global, for some reason share() doesnt want to work. This doesnt segfault, but also doesnt appear to work:

..
require Thread::Queue;
our $CONNECT : shared = 0;

5.8.1 says to the first:

Argument to share needs to be passed as ref
at telnetclient.perl line 103.

which is somewhat more helpful. Current solution:

require threads;
import threads;
require Thread::Queue;
import Thread::Queue;
import threads::shared;
share(\$CONNECT);

Which runs fine in both 5.8.0 and 5.8.1. Only difference being that 5.8.1 (release, rc4 and rc5), now segfaults upon exit, somewhere when trying to destroy a thread or threads ( gdb output from core:

#6  0x40350460 in Perl_ithread_destruct ()
#7  0x403505a4 in ithread_mg_free ()
#8  0x400a4e31 in Perl_sv_unmagic ()
#9  0x403515fa in Perl_ithread_DESTROY ()
#10 0x40352d41 in XS_threads_DESTROY ()
#11 0x4009d71b in Perl_pp_entersub ()

Fazit: How to make a script that runs with 5005threads, ithreads or none at all:

if($Config{use5005threads})
{
    debug("5005 Threads\n");
    require Thread;
    Thread->import('yield');
    require Thread::Queue; 
    import Thread::Queue;

    $conn = start_connection(1);
    $telnetprintqueue = Thread::Queue->new();
    $screenprintqueue = Thread::Queue->new();
    $connectqueue = Thread::Queue->new();
}
elsif($Config{useithreads})
{
    debug("I-Threads\n");
    require threads;
    import threads;
    require Thread::Queue; 
    import Thread::Queue;
    import threads::shared;
    share(\$CONNECT);
    share(\$PromptType);

    $telnetprintqueue = Thread::Queue->new();
    $screenprintqueue = Thread::Queue->new();    
    $conn = start_connection(2);
}
else
{
    debug("No Threads\n");
    $conn = start_connection(0);
}

.. and use Queues as much as possible.. :)

C.

[liz]

I must say I don't understand the problems you had with segfaults. I can't reproduce them, at least not with the piece of code given here.

I wonder though if this:

BEGIN {
    if ($Config{use5005threads}) { debug( "5005 Threads\n" );

        eval 'use Thread qw(yield); use Thread::Queue';
        die $@ if $@;

        $conn = start_connection( 1 );
        $telnetprintqueue = Thread::Queue->new;
        $screenprintqueue = Thread::Queue->new;
        $connectqueue     = Thread::Queue->new;

    } elsif ($Config{useithreads}) { debug( "I-Threads\n" );

        eval 'use threads; use threads::shared; use Thread::Queue';
        die $@ if $@;

        share( $CONNECT );
        share( $PromptType );

        $telnetprintqueue = Thread::Queue->new;
        $screenprintqueue = Thread::Queue->new;
        $conn = start_connection( 2 );

    } else { debug( "No Threads\n" );

        $conn = start_connection( 0 );
    }
}

actually works or breaks for you. I realize that in general you don't want string eval's, but these string eval's are happening once at compile time from a fixed string. And compile time itself is nothing but a string eval anyway. I guess that's the exception to the rule ;-)

Anyway, my approach feels cleaner to me, avoiding the forest of requires and imports.

Hope this helps.

Liz