Re: Request to detect the mistake in a perl script for finding inter-substring distance from a large text file

[lune]

As far as I can see there is no need to
a) read in the whole file at once
b) paste the lines together

Your count won't change if you do the counting line by line - which would solve your memory problem. So it would be worth looking at the part of your program that needs the whole file as string to see, whether this could be changed too. If not, there is the proposition about using "tie" already.

Then, you can search for all valid characters at once and use a hash to collect and count them.

This would be a possible solution:

#!/usr/bin/perl -w

use strict;
use warnings;
use diagnostics;
use Data::Dumper;

my $filename = "dna.txt";
open(my $fh, "<", $filename) || die "could not open $filename: $!\n";

my %bases;
my $cnt_errors = 0;

while (<$fh>) {
    # strip spaces
    s/\s+//ig;
    # collect results
    my @results = ($_ =~ /[ACGT]/ig);
    map { $bases{$_}++ } @results;

    $cnt_errors += ( length($_) - scalar @results );
}


print Dumper(%bases);
print "Errors: $cnt_errors\n";