Re^3: Memory usage while tallying instances of lines in a .txt file

[stevieb]

The following code does what you want, ie. "Strand" can be at any position on the first line, and it removes the extreme memory overhead of reading in the whole file at once.

use warnings;
use strict;

use Data::Dumper;
use List::Util qw(first);

my %hits;
my $index;

open my $fh, '<', 'file.txt' or die $!;

while (<$fh>){
    chomp;
    my @F = split ' ';

    if (/Strand/){
        $index = first { $F[$_] eq 'Strand' } 0..$#F;
        next;
    }

    if (! exists $hits{$F[$index+1]}{$F[$index+2]}) {
        $hits{$F[$index+1]}{$F[$index+2]}{'w'} = 0;
        $hits{$F[$index+1]}{$F[$index+2]}{'c'} = 0;
    }
    $hits{$F[$index+1]}{$F[$index+2]}{$F[$index]}++;
}

print Dumper \%hits;

Data used:

Strand
1   4   1   0
1   5   1   0
1   31  1   0
1   74  1   0

[TJCooper]

I'm not sure where I was reading the entire file into memory. Shouldn't these lines of code only handle the first line of the input file (given that they occur outside of the while-loop):

my @headers = split("\t",<$IN>);
my $index = first{$headers[$_] eq 'Strand'} 0..$#headers;

Indeed, your approach does not reduce RAM requirement.