Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change

Beware List::MoreUtils natatime()!

by samtregar (Abbot)
on Feb 09, 2009 at 07:20 UTC ( #742364=perlmeditation: print w/replies, xml ) Need Help??

I just got done tracking down a nasty memory leak due to my use of natatime(). It's not a function I've ever used before but it seemed convenient enough.

Run this code and watch the memory use grow:

use List::MoreUtils qw(natatime); my $count = 0; while (1) { my @rows = ("...") x 100; my $it = natatime(10,@rows); while(my @chunk = $it->()) { $count += @chunk; if ($count % 1000 == 0) { warn "COUNT : $count\n"; } } }

I haven't dug any deeper than this, so I'm not sure if it's something in List::MoreUtils XS code or a side-effect of using a closure for the iterator, or what. I'm also going to send this to the maintainer, so no need to remind me!

UPDATE: It would seem to be an XS bug - setting the environment var LIST_MOREUTILS_PP=1 makes it go away.

UPDATE 2: Corion asked, so I'll tell you - this is List::MoreUtils v0.22 on Perl v5.8.8. Oh, and the maintainer's email address doesn't work. Anybody want to take over and fix this?


Replies are listed 'Best First'.
Re: Beware List::MoreUtils natatime()!
by moritz (Cardinal) on Feb 09, 2009 at 09:21 UTC
    Others who are interested in List::MoreUtils' bugs will probably look at the bug tracker, so maybe you should open a ticket there.

    And FYI, there are already two other reports about memory leaks in that queue.

Re: Beware List::MoreUtils natatime()!
by JavaFan (Canon) on Feb 09, 2009 at 10:17 UTC
    It would seem to be an XS bug - setting the environment var LIST_MOREUTILS_PP=1 makes it go away.
    Not for me:
    $ LIST_MOREUTILS_PP=1 perl foo String found where operator expected at (eval 1) line 206, near "Carp: +:croak "each_arrayref: argument is not an array reference\n"" (Do you need to predeclare Carp::croak?) String found where operator expected at (eval 1) line 226, near "Carp: +:croak "each_array: unknown argument '$method' passed to iterator."" (Do you need to predeclare Carp::croak?) Use of inherited AUTOLOAD for non-method List::MoreUtils::natatime() i +s deprecated at foo line 14. Can't locate auto/List/MoreUtils/ in @INC (@INC contains: / +opt/perl/lib/5.10.0/i686-linux-64int-ld /opt/perl/lib/5.10.0 /opt/per +l/lib/site_perl/5.10.0/i686-linux-64int-ld /opt/perl/lib/site_perl/5. +10.0 /opt/perl/lib/site_perl/5.8.8 /opt/perl/lib/site_perl .) at foo +line 14
    It runs (with the memory leak) if I don't set LIST_MOREUTILS_PP.

      Your bug sounds like your version is depending on an accident of not-quite recent perls. It used to often "just work" to say Carp::croak '...';. That was accidental and whenever you see an exception like this, just make sure to load Carp in advance of loading the thing you're using.

      In your example, do:

      $ LIST_MOREUTILS_PP=1 perl -MCarp foo
        Looking into the source of List::MoreUtils, I find:
        require Carp; Carp::croak "some message";
        That only works if Carp was already loaded. In older versions of Perl, there was a common pragma (vars?) that loaded it, but that no longer is loaded in 5.10.

        Hence the error. And another reason to not use List::MoreUtils.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://742364]
Approved by grinder
Front-paged by Arunbear
[Corion]: Congratulations Lady_Aleena ;)
[Lady_Aleena]: Thanks Corion.
[Lady_Aleena]: This is one of my oldest accounts on the web. 8)
[Corion]: Lady_Aleena: Hehe - I think this is also my oldest active account on the web. I have an old university email address but that only gets spam nowadays :)
Discipulus arrived at work 1 hour late.. he do not noticed the clock bell
[Lady_Aleena]: My Wikipedia account is 3 years - 10 days younger.

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (8)
As of 2017-04-26 07:39 GMT
Find Nodes?
    Voting Booth?
    I'm a fool:

    Results (469 votes). Check out past polls.