Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Comment on

( #3333=superdoc: print w/ replies, xml ) Need Help??
Monks I come to you seeking your wisdom once again. I am having a bit of a memory management issue I was hoping you could help me with. I have written a script that produces a daily report on a system scan it does. It first reads in a benchmark file from the scan the day before into a hash. It then runs the daily scan dumping the info it finds into the new benchmark file and also compares the information to the data from the day before in the hash.

The problem I am running into is the data being read into the hash can be several hunder megabytes in size. I need to find a more efficent way to handle this data. Here is an example of the code I have written.
use strict; sub _read_benchmark { my %yesterday; open BENCH, "benchmark_file" or die "$!"; while (<BENCH>) { chomp; $yesterday{$_}++; } close BENCH or warn "$!"; return \%yesterday; } sub _scan_system { my $yesterday = shift; my %today; open BENCH, "> benchmark_file" or die "$!"; open IN, "find / $search_files -print |" or die "$!"; while (<IN>) { chomp; print BENCH "$_\n"; if (exists $yesterday->{$_}) { delete $yesterday->{$_}; } else { $today{$_}++; } } close IN or warn "$!"; close BENCH or warn "$!"; return \%today, $yesterday; } sub _print_report { ... } my $yesterday = _read_benchmark; my ($today, $yesterday) = _scan_system($yesterday); _print_report($today, $yesterday);
I believe there is a way I can tie the benchmark file to a hash and that would probably be a huge improvement, but I am not sure how to do that. Any suggestions on how to make this less of a memory hog would be very appreciated. Thanks!

-Prime

In reply to Memory Management Problem by PrimeLord

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • Outside of code tags, you may need to use entities for some characters:
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others exploiting the Monastery: (10)
    As of 2014-10-23 12:43 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?

      For retirement, I am banking on:










      Results (125 votes), past polls