Re: threads causing memory leak
by Corion (Patriarch) on Sep 11, 2010 at 13:11 UTC
|
You say "the code is called every 30 seconds" - do you mean you create a new thread every 30 seconds? Do you ever ->join that thread? Otherwise, I think you should use async. Personally, I would put the fetched results in a Thread::Queue and retrieve it from there in the main thread, so that all updating of new data happens within the main thread and only the fetching happens asynchronously, using ->dequeue_nb().
As Win32::GUI is event-driven, it might even make sense to have the worker thread send a window message to the main program to tell it to fetch the result.
| [reply] [d/l] [select] |
|
do you mean you create a new thread every 30 seconds?
Yes.
Do you ever ->join that thread?
No, i wait for it to finish.
Otherwise, I think you should use async.
async?
Update:
Using async does not help, same problem. joining the thread leads to:
Free to wrong pool 9f2520 not 3f4e10 at c:/strawberry/perl/vendor/lib/
+Win32/GUI.pm line 2703 during global destruction.
holli
You can lead your users to water, but alas, you cannot drown them.
| [reply] [d/l] [select] |
|
No, i wait for it to finish.
That's exactly what join does. But if you don't join it, it's memory is never returned to the memory pool.
Free to wrong pool 9f2520 not 3f4e10
That's a bug in Win32::GUI.
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
| [reply] |
|
|
|
|
Re: threads causing memory leak (No leak!)
by BrowserUk (Patriarch) on Sep 11, 2010 at 20:20 UTC
|
Here you go. Now tested. It fetches the url every 5 seconds ad nauseum, and no leaks.
#! perl -slw
use strict;
use threads;
use threads::shared;
use Thread::Queue;
use LWP::UserAgent;
sub t {
my $Q = shift;
while( $Q->dequeue ) {
my $ua = LWP::UserAgent->new;
my $response = $ua->get(
'http://www.mektek.net/mekmatch/listServersRss.mkz'
);
my $rss = $response->decoded_content
if $response->is_success;
print $rss if $rss;
print "done1";
}
}
my $Q = new Thread::Queue;
my $thread = threads->create( \&t, $Q );
require Win32::GUI;
my $mw = Win32::GUI::Window->new(
-width => 1000, -height => 800,
-title => "Test", -name => "Test",
-onTimer => sub {
$Q->enqueue( 1 );
}
);
my $use_up_some_memory = "x" x (1024 * 1024 * 20);
$mw->AddTimer( "T1", 5000 );
$mw->Show;
Win32::GUI::Dialog();
$Q->enqueue( undef );
$thread->join;
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
| [reply] [d/l] |
Re: threads causing memory leak
by nikosv (Deacon) on Sep 11, 2010 at 17:00 UTC
|
Free to wrong pool
I've been bitten by that as well when using Tk.It is an issue with reentrancy and/or thread safety.You might want to check this node out for more info Perl Tk and threads
You might also want to follow Corion's advise of sending a Window message back to the GUI thread since the call will be serialized in its message queue and thus won't break thread safety.
You might also run the app under Leakdiag and LDGraph which parses the data collected from Leakdiag and presents a nice diagram with the suspicious memory allocations so it might help you pinpointing the leak
| [reply] |
Re: threads causing memory leak
by zentara (Archbishop) on Sep 11, 2010 at 19:43 UTC
|
In Perl, there is this f*ing ref count thing, which cause complex, non-threadsafe modules, to not be completely destroyed, due to them still wanting to use a variable.
You can google for "perl memory thread safety" and see the gory details. My advice, if you want to play it safe, use a reusable-thread model in your software design. You create threads once, then reuse them, avoiding the need to worry about threads-based memory gains to the perl interpreter. So see Reusable threads demo and OS memory reclamation with threads on linux for examples.
| [reply] |
Re: threads causing memory leak
by BrowserUk (Patriarch) on Sep 11, 2010 at 13:11 UTC
|
If you post a runnable demo, I'd be happy to investigate.
| [reply] |
|
use strict;
use threads;
use warnings;
use Win32::GUI qw();
use LWP::UserAgent;
my $window = Win32::GUI::DialogBox->new( -title => "Test", -name => "T
+est", -onTimer => sub { return poll() } );
my $use_up_some_memory = join( "", "x" x (1024 * 1024 * 20) );
$window->AddTimer( "T1", 20000 );
$window->Show;
Win32::GUI::Dialog();
sub poll
{
my $rss : shared;
print "polling\n";
my $thread = threads->create( sub {
my $ua = LWP::UserAgent->new;
my $response = $ua->get( 'http://www.mektek.net/mekmatch/listS
+erversRss.mkz' );
$rss = $response->decoded_content
if $response->is_success;
} );
while ( $thread->is_running ) {
sleep(1); Win32::GUI::DoEvents;
}
return unless $rss;
}
holli
You can lead your users to water, but alas, you cannot drown them.
| [reply] [d/l] |
|
As I suspected it (and Corion called it), you constantly start new threads and never join or detach them, then they just hang around consuming memory once they've completed.
Also, as the last line of the thread sub is $rss, that is returned from the thread, consuming more memory. The fact that you've shared it, as well as returning it means you've consumed even more.
And while I'm critiquing, there are some other anomalies in your code:
- "x" x (1024 * 1024 * 20) is a single string of 20MB 'x's.
The join is utterly redundant.
- return unless $rss;
Return (nothing) unless $rss has some value.
Otherwise, just fall off the end of the subroutine and return ???
Try something like this (untested):
#! perl -slw
use strict;
use threads;
use Win32::GUI qw();
use LWP::UserAgent;
sub t {
my $rss;
my $ua = LWP::UserAgent->new;
my $response = $ua->get(
'http://www.mektek.net/mekmatch/listServersRss.mkz'
);
$rss = $response->decoded_content
if $response->is_success;
print "done1";
return $rss;
}
my $window = Win32::GUI::DialogBox->new(
-title => "Test", -name => "Test", -onTimer => sub { return poll()
+ }
);
my $use_up_some_memory = join( "", "x" x (1024 * 1024 * 20) );
$window->AddTimer( "T1", 5000 );
$window->Show;
Win32::GUI::Dialog();
sub poll
{
print "polling ... ";
my $thread = threads->create( \&t );
while ( $thread->is_running ) {
Win32::Sleep 10;
Win32::GUI::DoEvents;
}
print " done2";
return $thread->join;
}
Presumably you want to do something with the data returned from poll?
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
| [reply] [d/l] [select] |
|
I think you really need to do a $thread->join at the end of your subroutine.
But instead of polling, I would have the thread send a (Windows) message to the main thread to tell the main thread/window that it can now fetch the data and update things.
| [reply] [d/l] |
|
Re: threads causing memory leak
by Anonymous Monk on Sep 11, 2010 at 14:07 UTC
|
| [reply] |
Re: threads causing memory leak
by Anonymous Monk on Sep 11, 2010 at 13:09 UTC
|
This code is called every 30 seconds, and adds up roughly 20 megs of memory at each call. That of course leads to out of memory crashes pretty quickly.
Um, no, the code you showed only gets called once | [reply] |
Re: threads causing memory leak
by Proclus (Beadle) on Sep 12, 2010 at 11:44 UTC
|
| [reply] |
Re: threads causing memory leak
by holli (Abbot) on Sep 24, 2010 at 15:24 UTC
|
| [reply] [d/l] |
|
ha ha, I knew you would like timers, even if they are not Glib timers. :-)
| [reply] |