Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic

Comment on

( #3333=superdoc: print w/replies, xml ) Need Help??
Monks I come to you seeking your wisdom once again. I am having a bit of a memory management issue I was hoping you could help me with. I have written a script that produces a daily report on a system scan it does. It first reads in a benchmark file from the scan the day before into a hash. It then runs the daily scan dumping the info it finds into the new benchmark file and also compares the information to the data from the day before in the hash.

The problem I am running into is the data being read into the hash can be several hunder megabytes in size. I need to find a more efficent way to handle this data. Here is an example of the code I have written.
use strict; sub _read_benchmark { my %yesterday; open BENCH, "benchmark_file" or die "$!"; while (<BENCH>) { chomp; $yesterday{$_}++; } close BENCH or warn "$!"; return \%yesterday; } sub _scan_system { my $yesterday = shift; my %today; open BENCH, "> benchmark_file" or die "$!"; open IN, "find / $search_files -print |" or die "$!"; while (<IN>) { chomp; print BENCH "$_\n"; if (exists $yesterday->{$_}) { delete $yesterday->{$_}; } else { $today{$_}++; } } close IN or warn "$!"; close BENCH or warn "$!"; return \%today, $yesterday; } sub _print_report { ... } my $yesterday = _read_benchmark; my ($today, $yesterday) = _scan_system($yesterday); _print_report($today, $yesterday);
I believe there is a way I can tie the benchmark file to a hash and that would probably be a huge improvement, but I am not sure how to do that. Any suggestions on how to make this less of a memory hog would be very appreciated. Thanks!


In reply to Memory Management Problem by PrimeLord

Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":

  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?

    What's my password?
    Create A New User
    and all is quiet...

    How do I use this? | Other CB clients
    Other Users?
    Others contemplating the Monastery: (3)
    As of 2018-05-23 19:18 GMT
    Find Nodes?
      Voting Booth?