Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Comment on

( #3333=superdoc: print w/ replies, xml ) Need Help??

Using 5.8.2 (AS808) on XP, and processing a little over 200_000 files, I see a growth pattern of around 22k per iteration, or maybe 10 bytes per file.

If I fork each iteration of the search, the growth appears to be increased slightly to 31k/iter of 205428 files.

Doing a crude comparision of heap dumps taken before & after an iteration, it appears as if the leakage isn't due to something not being freed, but rather to fragmentation of the heap, as larger entities are freed and their space half re-used for smaller things, thereby requiring the heap to grow the next time the larger entity needs to be allocated.

Note: The comparision was very crude...with something like 12000 individual blocks on the heap, it had to be:)

Having the script exec itself after each iteration does stop the growth, but whether that is practical will depend upon the nature and design of your program.


Examine what is said, not who speaks.
"Efficiency is intelligent laziness." -David Dunham
"Think for yourself!" - Abigail
Timing (and a little luck) are everything!


In reply to Re: File::Find memory leak by BrowserUk
in thread File::Find memory leak by crabbdean

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • Outside of code tags, you may need to use entities for some characters:
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others meditating upon the Monastery: (14)
    As of 2014-11-21 19:10 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?

      My preferred Perl binaries come from:














      Results (114 votes), past polls