Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Perl seg fault while joining threads

by kamrul (Acolyte)
on Jul 07, 2015 at 21:45 UTC ( #1133644=perlquestion: print w/replies, xml ) Need Help??
kamrul has asked for the wisdom of the Perl Monks concerning the following question:

I have a code similar to the below. I have one main script which is calling another module named initial.pm. initial.pm opens up connection with an AMQP server (In my case RabbitMQ)and using Net::AMQP::RabbitMQ library for establishing the connection. Everything works fine except when I try to join my threads I get segmentation fault. I think the Net::AMQP::RabbitMQ is not thread safe. But this is only being used by the main thread. Im pretty sure you can reproduce the error if you just copy and past the codes below. main.pl
#!/usr/bin/perl use Cwd qw/realpath/; use File::Basename qw/dirname/; use lib 'lib'; use threads; use threads::shared; use initial; my @threads = (); my $run :shared = 1; my $init = load initial($name); $SIG{'TERM'} = sub { $run = 0; }; threads->create(\&proc1); threads->create(\&proc2); while($run){ sleep(1); print "I am main thread\n"; } $_->join() for threads->list(); sub proc1 { while($run){ sleep(1); print "I am child thread 1 \n" } } sub proc2 { while($run){ sleep(1); print "I am child thread 2 \n"; } }
lib/initial.pm
package initial; use Net::AMQP::RabbitMQ; use Cwd qw/realpath/; use File::Basename qw/dirname/; my $mq; my $stop = 0; sub load { my $class = shift; my $self = {}; connectq(); bless $self,$class; return $self; } sub connectq { $mq = Net::AMQP::RabbitMQ->new(); my ($host,$port,$user,$pass) = ('localhost','5672','guest','guest' +); $mq->connect($host, { user => $user, password => $pass, port => $port, timeout => 10, }); $mq->channel_open(1); $mq->consume(1, 'logger'); } 1;
If I call the initial class after creating the threads I dont see the error. It seems the threads are copying all the previously initiated variables into it when it is being created. In my case I dont need to access my $init variable from any other threads. So is there a way to tell perl not to copy a specific variable while creating new threads ?

Replies are listed 'Best First'.
Re: Perl seg fault while joining threads
by BrowserUk (Pope) on Jul 07, 2015 at 22:04 UTC
    So is there a way to tell perl not to copy a specific variable while creating new threads ?

    I don't have Net::AMQP::RabbitMQ, so this is untested, but if your description is accurate, it ought to work.

    Just re-structure your code slightly:

    #!/usr/bin/perl use Cwd qw/realpath/; use File::Basename qw/dirname/; use lib 'lib'; use threads; use threads::shared; use initial; my @threads = (); my $run :shared = 1; sub proc1 { while($run){ sleep(1); print "I am child thread 1 \n" } } sub proc2 { while($run){ sleep(1); print "I am child thread 2 \n"; } } threads->create(\&proc1); threads->create(\&proc2); my $init = load initial($name); $SIG{'TERM'} = sub { $run = 0; }; while($run){ sleep(1); print "I am main thread\n"; } $_->join() for threads->list();

    In general, it is better to start your threads before you create/allocate anything that is only used by your main thread.

    See threads: spawn early to avoid the crush. for more details.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
    I'm with torvalds on this Agile (and TDD) debunked I told'em LLVM was the way to go. But did they listen!
      There are some places in my code where I cant really create the thread before the instantiating class. Is there any other way ?
        Is there any other way ?

        Yes. If you spawn a thread early, it will not copy anything created after it is spawned. If you then arrange for it to spawn the threads that you need to spawn later, they will inherit the cleanliness of their parent. Ie. Won't duplicate anything created after that first thread was spawned.

        The code below looks complicated, but it is really quite straight forward. If you wrap it into a module and call it before you call your initial module, the api can be pretty transparent.

        This just demonstrates the technique; you can do the wrapping to suit your code/style:

        #! perl -slw use strict; use threads; use threads::shared; use Thread::Queue; my $run :shared = 1; sub func1 { print "func1: @_"; sleep 1 while $run; } sub func2 { print "func2: @_"; sleep 1 while $run; } my $Qspawn = new Thread::Queue; async { while( my $parms = $Qspawn->dequeue ) { my( $type, @args ) = split $;, $parms; if( $type == 1 ) { $Qspawn->enqueue( threads->new( \&func1, @args ) ); } elsif( $type == 2 ) { $Qspawn->enqueue( threads->new( \&func2, @args ) ) } else { warn "Unknown type: $type"; } } }->detach; ## now create/allocate your main thread stuff # require initial; # initial->import( entrypoints ); ## if needed # my $init = ... ## When you want to start func1 $Qspawn->enqueue( join $;, 1, 1, 'two', 3.3 ); my $thread1 = $Qspawn->dequeue; ## other stuff ## Now spawn func2 $Qspawn->enqueue( join $;, 2, 2, 'four', 8.8 ); my $thread2 = $Qspawn->dequeue; ## do stuff ## tell func1 & func2 to finish $run = 0; ## Join them $_->join for $thread1, $thread2; ## Let the spawner thread clean itself up exit; __END__ C:\test>spawner.pl func1: 1 two 3.3 func2: 2 four 8.8

        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
        I'm with torvalds on this Agile (and TDD) debunked I told'em LLVM was the way to go. But did they listen!

        Is there a reason that the call to 'initial' needs to be in the main thread? Could you not just put it in it's own thread and spawn that first?

Re: Perl seg fault while joining threads
by stevieb (Abbot) on Jul 07, 2015 at 21:50 UTC

    It's prudent to inform everyone when you've cross-posted, so that people can review what responses you got at the other site before re-inventing any wheels.

    -stevieb

Re: Perl seg fault while joining threads
by sundialsvc4 (Abbot) on Jul 07, 2015 at 22:54 UTC

    I wonder idly if Perl’s “copy-on-write” (COW) behavior could possibly be a factor here.   Perl would copy data-structures related to RabbitMQ into each child, even though the children weren’t the threads that had actually initialized it.   Therefore, if Rabbit has any sort of “destructor” subroutine, that destructor would see the initialized data-structures and of course try to clean it up.   I can see that there could be trouble brewing in that case ... closing library handles multiple times, closing (library) resources that this thread never allocated, and so on.   Even if Perl didn’t “double-free” anything, a library certainly could . . .

    BrowserUK, you’ve had plenty of experience with both Perl threads and libraries.   In your experience, does this Perl-specific COW behavior typically cause grief in cases like this one?   (And, BTW, I mean that as a serious, face-value question, addressed to an expert in such things.)

      Can locking be a solution ?
        Can locking be a solution ?

        No.

        But then, nothing in the post to which you've replied makes any sense. It is pure speculation by a totally discredited monk who as far as we can tell has never written a single line of working Perl code, much less anything that uses threads.

        He picks up terms like blotting paper and regurgitates them in interesting combinations much like the waffle generator. Except, he's old school; and does it all manually with his own artistic flair.

        On a more to the point note: If you were to post a more complete description of your problem (including the code, or a reference where I can view it if it is big), I might be able to offer you a solution to your problems.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
        I'm with torvalds on this Agile (and TDD) debunked I told'em LLVM was the way to go. But did they listen!

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1133644]
Approved by BrowserUk
help
Chatterbox?
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (11)
As of 2017-11-22 10:13 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    In order to be able to say "I know Perl", you must have:













    Results (317 votes). Check out past polls.

    Notices?