Re: Avoiding Memory Leaks

[kennethk]

Your recollection is correct: garbage collection should happen following both delete $dtgs{$dtg}; and $dtgs{$dtg} = \@files;, since both remove an array ref from %dtgs.

Depending on your array sizes, you may get more efficiency from a splice instead of your grep, since you need to create a new array and copy nearly all the old file names. That code might look something like

sub remove_from_dtgs {
    my ($dtg,$file) = @_;
    for my $i (reverse 0 .. $#files) {
        splice @{$dtgs{$dtg}}, $i, 1 if $dtgs{$dtg}[$i] eq $file;
    }
    delete $dtgs{$dtg} unless @{$dtgs{$dtg}};
}

If you know lists are unique (no repeats) and that this is the only routine that modifies the arrays, you can add some Loop Control and do a little better:

sub remove_from_dtgs {
    my ($dtg,$file) = @_;
    for my $i (reverse 0 .. $#files) {
        if ($dtgs{$dtg}[$i] eq $file) {
            splice @{$dtgs{$dtg}}, $i, 1;
            delete $dtgs{$dtg} unless @{$dtgs{$dtg}};
            last;
        }
    }
}

Note in the original, you missed parentheses in your test, and that all logical tests are scalar context, so the scalar is unnecessary.

Of course, this is an optimization, so make sure to actually test (perhaps with Devel::NYTProf) rather than guess at what's slow.

Update: Or, of course, given a uniqueness constraint, you could just use a hash:

open(my $dtg_file, "<", $infile) or die "Unable to open $infile: $!\n"
+;
while(<$dtg_file>) {
    chomp;
    my ($dtg,@files) = split /:/;
    $dtgs{$dtg}{$_}++ for @files;
}
close $dtg_file;

sub remove_from_dtgs {
    my ($dtg,$file) = @_;
    delete $dtgs{$dtg}{$file};
    delete $dtgs{$dtg} if !keys %{$dtgs{$dtg}}
}

[wink]

Thanks for your reply! The majority (something like 80%) of the DTGs have a single file associated with them, less than 1% have 4 or more, and only a few have 10+. So I don't think a splice will offer much of a performance benefit over the grep. Definitely will look into it thought, thanks! It's not a function I have used much

I also apologize for the formatting errors. I had to copy this by hand as it's on a non-internet-facing system.