Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Beware List::MoreUtils natatime()!

by samtregar (Abbot)
on Feb 09, 2009 at 07:20 UTC ( #742364=perlmeditation: print w/ replies, xml ) Need Help??

I just got done tracking down a nasty memory leak due to my use of natatime(). It's not a function I've ever used before but it seemed convenient enough.

Run this code and watch the memory use grow:

use List::MoreUtils qw(natatime); my $count = 0; while (1) { my @rows = ("...") x 100; my $it = natatime(10,@rows); while(my @chunk = $it->()) { $count += @chunk; if ($count % 1000 == 0) { warn "COUNT : $count\n"; } } }

I haven't dug any deeper than this, so I'm not sure if it's something in List::MoreUtils XS code or a side-effect of using a closure for the iterator, or what. I'm also going to send this to the maintainer, so no need to remind me!

UPDATE: It would seem to be an XS bug - setting the environment var LIST_MOREUTILS_PP=1 makes it go away.

UPDATE 2: Corion asked, so I'll tell you - this is List::MoreUtils v0.22 on Perl v5.8.8. Oh, and the maintainer's email address doesn't work. Anybody want to take over and fix this?

-sam

Comment on Beware List::MoreUtils natatime()!
Download Code
Re: Beware List::MoreUtils natatime()!
by moritz (Cardinal) on Feb 09, 2009 at 09:21 UTC
    Others who are interested in List::MoreUtils' bugs will probably look at the bug tracker, so maybe you should open a ticket there.

    And FYI, there are already two other reports about memory leaks in that queue.

Re: Beware List::MoreUtils natatime()!
by JavaFan (Canon) on Feb 09, 2009 at 10:17 UTC
    It would seem to be an XS bug - setting the environment var LIST_MOREUTILS_PP=1 makes it go away.
    Not for me:
    $ LIST_MOREUTILS_PP=1 perl foo String found where operator expected at (eval 1) line 206, near "Carp: +:croak "each_arrayref: argument is not an array reference\n"" (Do you need to predeclare Carp::croak?) String found where operator expected at (eval 1) line 226, near "Carp: +:croak "each_array: unknown argument '$method' passed to iterator."" (Do you need to predeclare Carp::croak?) Use of inherited AUTOLOAD for non-method List::MoreUtils::natatime() i +s deprecated at foo line 14. Can't locate auto/List/MoreUtils/natatime.al in @INC (@INC contains: / +opt/perl/lib/5.10.0/i686-linux-64int-ld /opt/perl/lib/5.10.0 /opt/per +l/lib/site_perl/5.10.0/i686-linux-64int-ld /opt/perl/lib/site_perl/5. +10.0 /opt/perl/lib/site_perl/5.8.8 /opt/perl/lib/site_perl .) at foo +line 14
    It runs (with the memory leak) if I don't set LIST_MOREUTILS_PP.

      Your bug sounds like your version is depending on an accident of not-quite recent perls. It used to often "just work" to say Carp::croak '...';. That was accidental and whenever you see an exception like this, just make sure to load Carp in advance of loading the thing you're using.

      In your example, do:

      $ LIST_MOREUTILS_PP=1 perl -MCarp foo
        Looking into the source of List::MoreUtils, I find:
        require Carp; Carp::croak "some message";
        That only works if Carp was already loaded. In older versions of Perl, there was a common pragma (vars?) that loaded it, but that no longer is loaded in 5.10.

        Hence the error. And another reason to not use List::MoreUtils.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://742364]
Approved by grinder
Front-paged by Arunbear
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (14)
As of 2014-09-23 14:55 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (223 votes), past polls