Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Re: mem usage

by almut (Canon)
on May 26, 2010 at 17:47 UTC ( #841767=note: print w/ replies, xml ) Need Help??


in reply to mem usage

Perl data structures (see PerlGuts Illustrated) need more memory than the mere user data they hold.

A quick check shows that reading 1_000_000 empty lines into an array (the way you do it) leads to a process size of 192MB on my machine (with Perl 5.10.1).  So if you have 11_000_000 non-empty ones...

Update: using ikegami's suggestion, the memory usage for the same 1_000_000 empty lines reduces to 78MB without pre-extending, and (interestingly) 86MB with pre-extending the array ($#lines = 1_000_000).   Update2: and 93MB with just $#lines = 100_000 (??)

Update3: with @lines = (), pre-extending the array no longer increases the memory requirements. But it doesn't help to reduce it either (presumably, it just improves speed).


Comment on Re: mem usage
Select or Download Code
Re^2: mem usage
by halfcountplus (Hermit) on May 26, 2010 at 18:05 UTC
    Perl data structures (see PerlGuts Illustrated) need more memory than the mere user data they hold.

    Will read. I presumed they used a bit more space, say a few pointers per element for an array -- but that is really really quite a lot.

    I've also read somewhere that perl allocates in "power of 2" chunks. So when you add a 2nd element, you get allocated space for 4, then at 5 you get 8, at 9 you get 16 and so on. Is this true?

      Close. When it grows, the array size doubles in size plus 4. An array with 1000 scalars in it could take up as much as

      Array overhead + ( 999*2+4 ) * size of a pointer + 1000 * size of the scalars
Re^2: mem usage
by ikegami (Pope) on May 26, 2010 at 18:12 UTC
    There was a bug causing the array to have twice the desired number of elements. I don't know if you were affected by it.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://841767]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (5)
As of 2014-12-21 04:09 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (102 votes), past polls