Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

ithreads memory leak

by DNAb (Novice)
on Apr 08, 2015 at 03:38 UTC ( #1122762=perlquestion: print w/replies, xml ) Need Help??
DNAb has asked for the wisdom of the Perl Monks concerning the following question:

I'm puzzled by something that seems simple, but yet is yielding strange results. I'm using threading as part of a decent sized program, the easiest way to describe the purpose of this code is it's part of an alarm system.

What I'm running into is a memory leak when using ithreads, sleep, and detach. This doesn't seem to happen if I use join to wait for the thread to finish execution, but I can't do this because the main loop has to keep running.

Here is a short snippet of code that exhibits the issue, the modules are in there for two reasons, one being it shows the leak better, two being these are what are used in the production code.

use warnings; use IO::Handle; use threads; use strict; use DBI; use Email::MIME; use Email::Sender::Simple qw(sendmail); while (<STDIN>) { chomp(my $msg = $_); print "Received: $msg\n"; if ($msg =~ /start/) { my $thread = threads->create(\&thread); } } sub thread { print "Thread started!\n"; sleep 5; threads->detach(); }

I'm using PERL 5.20.2, compiled using perlbrew. I'm using top and ps to watch memory usage on a Linux server. Type "start" a few times and the memory usage balloons about 6MB each time. At random points the ballooning stops, and PERL will reclaim seemingly about 1MB of memory, then start ballooning again.

What am I missing here?

Replies are listed 'Best First'.
Re: ithreads memory leak
by BrowserUk (Pope) on Apr 08, 2015 at 04:27 UTC
    the memory usage balloons about 6MB each time. At random points the ballooning stops, and PERL will reclaim seemingly about 1MB of memory, then start ballooning again.

    What happens if you recast the code as follows?:

    use warnings; use IO::Handle; use threads stack_size => 64*1024; use strict; use DBI; use Email::MIME; use Email::Sender::Simple qw(sendmail); while (<STDIN>) { chomp(my $msg = $_); print "Received: $msg\n"; if ($msg =~ /start/) { threads->create(\&thread)->detach; } } sub thread { print "Thread started!\n"; sleep 5; }

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
    In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked

      Same effect unfortunately, to confirm my code now looks like:

      #!/home/alerting/perl5/perlbrew/perls/perl-5.20.2/bin/perl use warnings; use IO::Handle; use threads stack_size => 64*1024; use strict; use DBI; use Email::MIME; use Email::Sender::Simple qw(sendmail); while (<STDIN>) { chomp(my $msg = $_); print "Received: $msg\n"; if ($msg =~ /start/) { threads->create(\&thread)->detach; } } sub thread { print "Thread started!\n"; sleep 5; }

      Memory usage still balloons, stops, then starts up again. Very puzzling, at least to me.

        I don't believe you are seeing a leak. I have to express it this way because I don't have *nix to test on; but running your code on my system, I see memory growth, but not a leak.

        To confirm my findings, tweak your code so:

        #!/home/alerting/perl5/perlbrew/perls/perl-5.20.2/bin/perl use warnings; use IO::Handle; use threads stack_size => 64*1024; use strict; use DBI; use Email::MIME; use Email::Sender::Simple qw(sendmail); while (<STDIN>) { threads->create(\&thread)->detach; sleep 5; } sub thread { print "Thread started!\n"; sleep 4; }

        Essentially what that does is limit your program to using only child thread at a time. (Not useful in the real world, but good for testing.).

        Run that and hold the enter key down while monitoring the memory usage and you should see an initial jump from circa 7MB to around 13MB, and then the memory usage should stay steady no matter how long you hold the key down for. Come back and let me know if that is true; and then I'll try and explain what you are seeing.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
        In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked
Re: ithreads memory leak
by sundialsvc4 (Abbot) on Apr 08, 2015 at 23:37 UTC

    Truer words have never been spoken than what BrowserUK just so-well said:   the key is to limit the number of threads that are active at any one time, and to separate this from the number of requests amount of work that they have to do.   Even if you receive (say ...) 1,000 start messages in the space of (say ...) one millisecond, you must not attempt to “therefore ...” launch 1,000 threads.   Instead, some of those requests will briefly have to wait-their-turn.

    You should have a pool of “worker threads,” of some configurable size that will not exceed this system’s capacity.   All of those threads, however many there may be, should be waiting for a message (start ...) to arrive on a common, thread-safe queue.     Meanwhile, your main thread, instead of “starting a new thread” each time such a message arrives, should instead be posting the message to that queue.   When such a message arrives for them, they carry out the work, then wait again.   (The main-thread is the only “writer.”   All of the workers are “readers.”)   If any worker is waiting, it will get the message without delay.   If all of the workers are busy, the message will briefly sit in the queue until some worker can receive it.   Nevertheless, the total number of threads that the operating system is being asked to support ... or, to provide memory for ... will never grow beyond the proscribed limits, no matter how stuffed-up the queue (momentarily) gets.

    And, there’s already a lot of CPAN code out there to help you, so be sure not to re-invent the wheel here.   (Say ...) Thread::Pool, Thread::Queue, Thread::Signal . . .

    You should, of course, design the threads so that they dispose of all storage needed to service their latest request, before they go-to-sleep waiting for another request to arrive.   If you take care to do that, Perl’s very-clever memory manager should automagically take care of the rest.   Yes, the memory-size of the parent process may grow to be large, but it should not grow uncontrollably leak.

      I see what you are both saying, I'm just not 100% sure how to make this code any different right now. It's somewhat unique, but it's still totally in beta so I'm fine with revisiting most aspects.

      The use case is as follows: An assortment of showcases are alarmed. To access a showercase a user must press a function key on a keypad, this starts a timer which gives them up to three minutes to open the showcase. If the user doesn't open the showcase, update the database to require another key press, if the user does open the showcase update the point as open in the database and then kill the walking timer, and start the open timer. The open timer lets them keep the door open for ten minutes, if this timer expires then send an alert, otherwise when the user closes the door kill the open timer. I realize this is complicated, I'm trying to give anyone reading this thread an idea of why it is how it is.

      Anyway, as promised here is the code:

      (Fair warning, by anyone's standards it's probably ugly)

        First:Ignore anything and everything sundialsvc4 says. He has a proven track record of meaningless, misdirected & pure malicious posts on subjects he has provably no knowledge or understanding of. And in particular, threads.

        then kill the walking timer,

        The fundamental problem with your code is this:

        #! perl -slw use strict; use threads; sub sleeper { print "thread started; sleeping"; $SIG{'KILL'} = sub { print "Thread dieing"; threads->exit(); }; sleep( 1000 ); print "Thread dieing"; threads->detach; } my $t = threads->new( \&sleeper ); sleep 5; print "Killing thread"; $t->kill( 'KILL' )->detach; print "thread status: ", $t->error(); sleep 100 while 1;

        Run that code and watch the process (top or whatever). It immediately creates a thread that just sleeps for a thousand seconds. The main thread then waits for 5 seconds and then (attempts to) kills the sleeping thread. But the thread never sees the "signal" ... until it finishes sleeping.

        This is because those "signals" aren't real signals; but some home-brewed simulation of them that cannot even interrupt a sleep. They (the pseudo-signals), should never have been added to the threads api; and they should never be used!

        And that explains the "memory growth" problem; you are expecting these threads to go away when before you start their replacements; but they simply won't.

        And that's a problem for your architecture which is built around that premise.

        I have 3 solutions for you: one is a quick fix; and the other two will required fairly extensive changes to your program.

        (But its my dinner time right now, so I'll detail them in a while. I just wanted to warn you to ignore sundialsvc4 who has never posted a single line of working code here. Be warned!)


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
        In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1122762]
Approved by BrowserUk
help
Chatterbox?
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (7)
As of 2018-01-17 11:04 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    How did you see in the new year?










    Results (198 votes). Check out past polls.

    Notices?