http://www.perlmonks.org?node_id=841776


in reply to Re^2: mem usage
in thread mem usage

Woops, what I posted is very buggy. Should be 
my @lines;
$#lines = 12_000_000;
@lines = ();                 <------ was missing
push @lines, $_ while <IN>;

[download]
$ perl -e'print "abcdef\n" for 1..11_000_000' |
   perl -E'
      $#a=11_000_000;
      @a=();
      push @a, $_ while <>;
      say int(`ps --no-heading -o vsz $$`/1000)
'
480

[download]

480MB for 77MB file with 11 million lines.

Replies are listed 'Best First'.
Re^4: mem usage
by halfcountplus (Hermit) on May 26, 2010 at 18:40 UTC
    Hmmm, yeah that's much better. What's the significance of this: 
          @a=();

    [download]
    After presizing -- I would have thought that would just clear the array?
[reply]
[d/l]

      It does indeed clear the array, but it does not free it the underlying buffer.

      Setting $#a extends the visible size of the array, not just the internal one. The elements I proceeded to push onto the array were getting added after the 12 mil empty ones.

      @a=(); empties the visible array, but doesn't deallocate the internal buffer like undef @a; does. 

      $ perl -MDevel::Peek -e'
   my @a = qw( a b c ); Dump(\@a,1);
   @a=();               Dump(\@a,1);
   undef(@a);           Dump(\@a,1);
'
SV = IV(0x816ce54) at 0x816ce58
  REFCNT = 1
  FLAGS = (TEMP,ROK)
  RV = 0x817bc50
  SV = PVAV(0x816d038) at 0x817bc50
    REFCNT = 2
    FLAGS = (PADMY)
    ARRAY = 0x817b500
    FILL = 2                <---- 3 elements
    MAX = 3                 <---- 4 slots
    ARYLEN = 0x0
    FLAGS = (REAL)
SV = IV(0x816c154) at 0x816c158
  REFCNT = 1
  FLAGS = (TEMP,ROK)
  RV = 0x817bc50
  SV = PVAV(0x816d038) at 0x817bc50
    REFCNT = 2
    FLAGS = (PADMY)
    ARRAY = 0x817b500
    FILL = -1              <---- 0 elements
    MAX = 3                <---- 4 slots
    ARYLEN = 0x0
    FLAGS = (REAL)
SV = IV(0x816c154) at 0x816c158
  REFCNT = 1
  FLAGS = (TEMP,ROK)
  RV = 0x817bc50
  SV = PVAV(0x816d038) at 0x817bc50
    REFCNT = 2
    FLAGS = (PADMY)
    ARRAY = 0x0            <---- No array allocated
    FILL = -1              <---- 0 elements
    MAX = -1               <---- 0 slots
    ARYLEN = 0x0
    FLAGS = (REAL)

      [download]

      If you don't want to rely on that, you could do 

      my @lines;
$#lines = 12_000_000;
$lines[$.-1] = $_ while <>;
$#lines = $.-1;

      [download]
[reply]
[d/l]
[select]
        Devel::Peek looks interesting. Thanks again.
[reply]

In Section Seekers of Perl Wisdom