Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

Re^3: mem usage

by ikegami (Pope)
on May 26, 2010 at 18:10 UTC ( #841776=note: print w/ replies, xml ) Need Help??


in reply to Re^2: mem usage
in thread mem usage

Woops, what I posted is very buggy. Should be

my @lines; $#lines = 12_000_000; @lines = (); <------ was missing push @lines, $_ while <IN>;
$ perl -e'print "abcdef\n" for 1..11_000_000' | perl -E' $#a=11_000_000; @a=(); push @a, $_ while <>; say int(`ps --no-heading -o vsz $$`/1000) ' 480

480MB for 77MB file with 11 million lines.


Comment on Re^3: mem usage
Select or Download Code
Re^4: mem usage
by halfcountplus (Hermit) on May 26, 2010 at 18:40 UTC
    Hmmm, yeah that's much better. What's the significance of this:
    @a=();
    After presizing -- I would have thought that would just clear the array?

      It does indeed clear the array, but it does not free it the underlying buffer.

      Setting $#a extends the visible size of the array, not just the internal one. The elements I proceeded to push onto the array were getting added after the 12 mil empty ones.

      @a=(); empties the visible array, but doesn't deallocate the internal buffer like undef @a; does.

      $ perl -MDevel::Peek -e' my @a = qw( a b c ); Dump(\@a,1); @a=(); Dump(\@a,1); undef(@a); Dump(\@a,1); ' SV = IV(0x816ce54) at 0x816ce58 REFCNT = 1 FLAGS = (TEMP,ROK) RV = 0x817bc50 SV = PVAV(0x816d038) at 0x817bc50 REFCNT = 2 FLAGS = (PADMY) ARRAY = 0x817b500 FILL = 2 <---- 3 elements MAX = 3 <---- 4 slots ARYLEN = 0x0 FLAGS = (REAL) SV = IV(0x816c154) at 0x816c158 REFCNT = 1 FLAGS = (TEMP,ROK) RV = 0x817bc50 SV = PVAV(0x816d038) at 0x817bc50 REFCNT = 2 FLAGS = (PADMY) ARRAY = 0x817b500 FILL = -1 <---- 0 elements MAX = 3 <---- 4 slots ARYLEN = 0x0 FLAGS = (REAL) SV = IV(0x816c154) at 0x816c158 REFCNT = 1 FLAGS = (TEMP,ROK) RV = 0x817bc50 SV = PVAV(0x816d038) at 0x817bc50 REFCNT = 2 FLAGS = (PADMY) ARRAY = 0x0 <---- No array allocated FILL = -1 <---- 0 elements MAX = -1 <---- 0 slots ARYLEN = 0x0 FLAGS = (REAL)

      If you don't want to rely on that, you could do

      my @lines; $#lines = 12_000_000; $lines[$.-1] = $_ while <>; $#lines = $.-1;
        Devel::Peek looks interesting. Thanks again.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://841776]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (8)
As of 2014-12-26 04:00 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (165 votes), past polls