Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask
 
PerlMonks  

Re^8: memory use array vs ref to array

by choroba (Cardinal)
on Sep 18, 2016 at 12:26 UTC ( [id://1172051]=note: print w/replies, xml ) Need Help??


in reply to Re^7: memory use array vs ref to array
in thread memory use array vs ref to array

I don't know what $GOTPS is. But I've run 
#!/usr/bin/perl
use warnings;
use strict;
use feature qw{ say };

say $$;
sleep 10;

my $file = shift;
open my $FH, '<', $file or die $!;

my $string;

{ local $/; $string = <$FH> };
say length $string;
sleep 1;

undef $string;

seek $FH, 0, 0;
$string = do { local $/; <$FH> };
say length $string;

[download]

I've used the PID reported by the script in another terminal and ran: 

while : ; do ps -o vsize,rss -p $PID | grep '[0-9]' ; done

[download]

And got the following output: 

 17504  3924
 17504  3924
 17504  3924
 17504  3924
 17504  3924
 17504  3924
1970632 20024
1970632 46648
1970632 71224
1970632 95920
1970632 122544
1970632 147120
1970632 171696
1970632 198320
1970632 224944
1970632 249520
1970632 280240
1970632 304816
1970632 331440
1970632 358064
1970632 384688
1970632 411312
1970632 442032
1970632 468656
1970632 493232
1970632 519856
1970632 546480
1970632 573104
1970632 599728
1970632 626352
1970632 652976
1970632 679600
1970632 706224
1970632 732848
1970632 759472
1970632 786096
1970632 812720
1970632 837296
1970632 865968
1970632 890544
1970632 917168
1970632 945840
1970632 972464
1970632 999088
1970632 1025712
1970632 1052336
1970632 1078960
1970632 1105584
1970632 1132208
1970632 1156784
1970632 1183408
1970632 1210032
1970632 1236656
1970632 1263280
1970632 1289904
1970632 1316528
1970632 1341104
1970632 1367728
1970632 1394352
1970632 1420976
1970632 1447600
1970632 1472176
1970632 1502896
1970632 1529520
1970632 1556144
1970632 1582768
1970632 1607344
1970632 1633968
1970632 1660592
1970632 1685004
1970632 1695232
1970632 1705000
1970632 1713976
1970632 1724008
1970632 1732720
1970632 1742488
1970632 1751464
1970632 1760704
1970632 1769944
1970632 1779184
1970632 1788952
1970632 1798192
1970632 1807960
1970632 1817728
1970632 1827232
1970632 1836472
1970632 1846768
1970632 1856800
1970632 1866832
1970632 1876600
1970632 1886368
1970632 1896136
1970632 1906168
1970632 1916464
1970632 1926760
1970632 1937056
1970632 1947088
1970632 1956856
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 1957120
1970632 242476
1970632 203684
1970632 100272
1970632 4480
1970632 28328
1970632 52904
1970632 77480
1970632 104104
1970632 128680
1970632 153264
1970632 183984
1970632 208560
1970632 235184
1970632 259760
1970632 290480
1970632 315056
1970632 341680
1970632 368304
1970632 394928
1970632 421552
1970632 448176
1970632 472752
1970632 499376
1970632 530096
1970632 560816
1970632 587440
1970632 614064
1970632 640688
1970632 665264
1970632 691888
1970632 718512
1970632 745136
1970632 771760
1970632 802480
1970632 829104
1970632 857776
1970632 884400
1970632 911024
1970632 937648
1970632 964272
1970632 988848
1970632 1015472
1970632 1040048
1970632 1068720
1970632 1095344
1970632 1121968
1970632 1148592
1970632 1179312
1970632 1210032
1970632 1234608
1970632 1261232
1970632 1287856
1970632 1318576
1970632 1345200
1970632 1371824
1970632 1398448
1970632 1425072
1970632 1451696
1970632 1476272
1970632 1502896
1970632 1529520
1970632 1556144
1970632 1582768
1970632 1611440
1970632 1638064
1970632 1662640
1970632 1685840
1970632 1696860
1970632 1705836
1970632 1715868
1970632 1725900
1970632 1735404
1970632 1744380
1970632 1754412
1970632 1763124
1970632 1773156
1970632 1782132
1970632 1791636
1970632 1801668
1970632 1811436
1970632 1820676
1970632 1830444
1970632 1840476
1970632 1849980
1970632 1861068
1970632 1871364
1970632 1881660
1970632 1893012
1970632 1902780
1970632 1912548
1970632 1922580
1970632 1932876
1970632 1942116
1970632 1951884
1970632 240492
1970632 169964
1970632 85324

[download]

($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord
}map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,

[download]

Replies are listed 'Best First'.
Re^9: memory use array vs ref to array
by Anonymous Monk on Sep 18, 2016 at 22:00 UTC

    :) whoops 

    use File::Which qw/ which /;
our $GOTPS = which('ps');

    [download]

    I've used the PID reported by the script in another terminal and ran:

    groan :) thats too much lines 

    #!/usr/bin/perl --
use warnings;
use strict;
use feature qw{ say };
use File::Which qw/ which /;
our $GOTPS = which('ps');

say $$;
psMem();

my $file = shift;
open my $FH, '<', $file or die $!;

my $string;

{ local $/; $string = <$FH> };
say length $string;

psMem(); 

undef $string;
psMem(); 

seek $FH, 0, 0;
$string = do { local $/; <$FH> };
say length $string;

psMem();
undef $string;
psMem(); 


sub psMem {
    my( $pid ) = @_;
    my @lines = qx{$GOTPS   -o vsize,rss     -p $pid };
    chomp @lines;
    foreach my $line (@lines) {
        my ( $vsize, $rss) = split /\s+/, $line;
        return "VM: $vsize   RSS: $rss  ";
    }
    return;
}

    [download]
[reply]
[d/l]
[select]
      You don't send any parameter to psMem . Also, you never output anything related to memory consumption. But, most importantly, it can still miss the allocation, as BrowserUk say the memory "gets released directly back to OS":

      > The reason is that in the later case, the data is read into an internal mortal temporary scalar; and then copied from there to the named lexical, before the memory attached to the temp is freed. As the allocation is greater than (from memory) 1MB, (on windows at least) such huge allocations are allocated directly from the OS's virtual memory rather than from the process' heap; and then get released directly back to OS.

      After having fixed the code, I ran your code with the following output: 

      6378
VM:    RSS: VSZ  
VM:    RSS: 17780  
2000000000
VM:    RSS: VSZ  
VM: 1970908   RSS: 1957712  
VM:    RSS: VSZ  
VM:    RSS: 17780  
2000000000
VM:    RSS: VSZ  
VM: 4412316   RSS: 3910984  
VM:    RSS: VSZ  
VM: 1970908   RSS: 1957856

      [download]

      Wait, it seems the memory consumption is indeed doubled! I checked with my previous code to realize I was running it on a different machine with different Perl version. I tried with blead, too, and it seems memory is doubled on Linux in 5.18, but not in 5.20 and later.

      ($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord
}map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,

      [download]
[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?

www.com | www.net | www.org
Node Status?
node history
Node Type: note [id://1172051]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? • Last hour • Other CB clients
Other Users?
Others surveying the Monastery: (6)
As of 2024-04-19 10:07 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found