Re: help with memory leak

"Leaked SVs are SVs which are not released after the end of the scope they have been created. These SVs include global variables and internal caches. For example, if you call a method in a tracing block, perl might prepare a cache for the method. Thus, to trace true leaks, no_leaks_ok() and leaks_cmp_ok() executes a block more than once."

When I tried your code with leaktrace(), a large number of leaks (~50) were detected; however, with no_leaks_ok(), I got none. This could mean I am unable to reproduce your problem. Please advise what OS and Perl version you're using. Can you also show your testing code. I'm using:

$ perl -v
This is perl 5, version 14, subversion 2 (v5.14.2) built for darwin-th
+read-multi-2level

$ uname -a
Darwin ganymede 11.4.2 Darwin Kernel Version 11.4.2: Thu Aug 23 16:25:
+48 PDT 2012; root:xnu-1699.32.7~1/RELEASE_X86_64 x86_64
[download]

The most obvious source of a memory leak in the code you posted is in the innermost loop, where you declare an array and take a reference to it. Every iteration through that innermost loop creates a new array with its own memory; this memory will not be released until %haystack goes out of scope (you don't show that in your code). So, @quartic and @derivative are both created afresh in the innermost loop and their memory should be freed after exiting that loop; however, multiple @zeros are created and their memory is not released because a reference is stored in %haystack.

I'm going to suggest a partial solution whereby you don't create any arrays at all in the innermost loop; its partial because I don't know what you're doing with %haystack. The fact that you're not declaring and then assigning any arrays may give you a performance bonus: Benchmark will tell you.

foreach my $x (3 .. $rep ) {
    foreach my $y (2 .. $x-1 ) {
        foreach my $z (1 .. $y-1 ) {
            # expanded form of x*(x - a)*(x - b)*(x - c)
            # coeffs are in an array
            $haystack{"$x, $y, $z"} =  [
                poly_roots(
                    poly_derivative(
                        1, -$x - $y - $z, $x*$y + $x*$z + $y*$z, -$x*$
+y*$z, 0
                    )
                )
            ];
        }
    }
}
[download]

[Side issue: You'll note I've changed $a, $b and $c to $x, $y and $z. $a and $b are special package variables and it's generally best not to use them except for their intended purpose (see perlvar for more details). $c is not special: I just changed that also for completeness. $x, $y and $z may not be the best choices (but $a and $b and definitely bad choices) — I'll leave you to decide what you want to call them.]

It would be helpful if you provided a self-contained script that showed:

What sort of values are assigned to $rep
How and where you're using %haystack (beyond the assignment you show)
What tests you're running

If your output is particularly lengthy, you can wrap it in <spoiler>...</spoiler> or <readmore>...</readmore> tags (see Markup in the Monastery for details). For your self-contained script, you'll find guidelines in: How do I post a question effectively?

-- Ken

Comment on Re: help with memory leak Select or Download Code

Replies are listed 'Best First'.
Re^2: help with memory leak by crunch_this! (Acolyte) on Apr 17, 2013 at 19:44 UTC
Looks like I've got a slightly different version: `perl -v This is perl 5, version 14, subversion 2 (v5.14.2) built for MSWin32-x +86-multi-thread` [download] uname -a didn't give me anything. I don't know what I did wrong Do you want to know what leaktest -verbose has to say? If there's a way to copy/paste it rather than type it by hand it would be great to know... What sort of values are assigned to $rep In this question & in testing I use small values (like <100) just to see how well it works. Once it's polished I want to use numbers basically a big as my computer can handle, like 1000 or bigger. So I expect the haystack to be huge even without any memory leaks. & if my computer can't handle that I'll go to some cloud computing thing to do it. What tests you're running In addition to the ones mentioned already, I've tried is_cycle_ok(), weaken() & I don't remember what else. How and where you're using %haystack (beyond the assignment you show) I was thinking of creating all polynomials with integer solutions, whose derivatives also have integer solutions. So a hash where the keys are the solutions of the polynomials and corresponding values are the solutions of the derivatives. The subroutine below searches the derivatives' zeros for zero-sets that only contain "approximte" integers. Then a hash slice gets created with the keys I want & Dumper prints it out. Here are the threads http://perlmonks.org/?node_id=1027571 http://perlmonks.org/?node_id=1028273 Here's the entire program, finished with assistance of user hdb who replied earlier in this node, & after changing a, b & c to x, y & z: Read more... (3 kB)	[reply] [d/l] [select]
Re^3: help with memory leak by kcott (Archbishop) on Apr 18, 2013 at 05:16 UTC
Don't worry about `uname`. What you've shown indicates we're using the same Perl version but completely different OSes: I'm using Mac OS X; you're using MS Windows. Be aware how the number of iterations grows with respect to `$rep`: 4 = 4; 5 = 10; 10 = 120; 20 = 1140; 100 = 161700; and so on. You didn't indicate how the code changes affected the leaks. My tests still show no leaks — I'm unable to reproduce your problem. I can't see any leak-related isues with `%haystack`; although, that doesn't mean there aren't any. I do note that you're populating `@wants` twice with what appears to be the same data: `my @wants = grep { is_approximately_an_integer( @$_ ) } values %haysta +ck; # intervening comments here only @wants = grep { is_approximately_an_integer( @{$haystack{$_}} ) } keys + %haystack;` [download] Obviously, you only want one of those. Also, if `%haystack` is only used to populate `@wants` (you don't show any other usage in your code), consider whether the `is_approximately_an_integer()` filter might be better placed in the innermost loop (possibly doing away with `%haystack` altogether). -- Ken	[reply] [d/l] [select]
Re^4: help with memory leak by crunch_this! (Acolyte) on Apr 18, 2013 at 18:31 UTC
Changing the code doesn't change whether or not leaktrace detects any leaks, including putting the subroutine in the innermost loop. How could I do this without %haystack, since if there are actually no memory leaks I think that would probably be much less of a drag on my computer? & simpler is better imho. I'm open to anything that would work. The function no_leaks_ok says there aren't any leaks so I'm with you there but leaktrace -verbose says Read more... (2 kB)	[reply] [d/l]
Re^5: help with memory leak by kcott (Archbishop) on Apr 18, 2013 at 19:37 UTC
Re^4: help with memory leak by crunch_this! (Acolyte) on Apr 18, 2013 at 21:14 UTC
I'm suggesting you do something like this (which does away with %haystack altogether): `my @wants; ... push @wants, grep { is_approximately_an_integer( @$_ ) } [ poly_roots( ...` OK that works much better, more efficient & no problems of any kind with leaktrace or no_leaks_ok, etc. The reason I thought a hash would be a good idea was that I'm also interested in getting back the corresponding $x, $y, $z at the end. That's why the coeffecients look the way they do in the poly_derivative definition. Is there a way to do that without using a hash? If not I'd still gladly trade that for more efficiency because I can always integrate what I get, or I could just work with the coefficients rather than the zeros.	[reply] [d/l]
Re^5: help with memory leak by kcott (Archbishop) on Apr 18, 2013 at 21:58 UTC
Re^6: help with memory leak by crunch_this! (Acolyte) on Apr 18, 2013 at 23:17 UTC