Help to improve script that searches a file for key words in a second file.

[mastarr]

Hello everybody,

I'm searching a file of annotations for a select list of terms in a second file. It seems to work fine but I'd like to learn alternative ways to do it or learn how to improve it. Specifically, is there is a way I can improve the regular expressions in this line:

 /(\b($term)\b|$term-)\b|(-$term)\b/i) {

Thanks!

Here is the full script:

#!/usr/bin/perl
# Read search terms into array @terms
open (SRC, $searchTermsFile) || die "can't open search terms file";
@terms = <SRC>;
chomp (@terms);
@terms = sort @terms;
my @annotationlist;
#-------------------------------------------------------
# Search each line in annotations file for match with search terms
%nTermHits = ();
$nTotalHits = 0;
open (ANN, $annotationsFile) || die "$annotationsFile won't open\n";
while ($line = <ANN>) {
    #chomp($line);
    @fields = split(/\t/, $line);
    $annotation = $fields [2];
    foreach $term (@terms) {
        if($annotation =~ /(\b($term)\b|$term-)\b|(-$term)\b/i) {
            $nTermHits {$term}++;
            $nTotalHits++;
            push(@annotationlist, "$annotation\n");
    }
}}
#@annotationlist = sort @annotationlist;
#print "@annotationlist\n";
#---------------------------------------
foreach $term (@terms) {
    if ($nTermHits {$term} > 0) {
        printf "%s\t%d\n", $term, $nTermHits {$term};
    }
}
print "Total hits: $nTotalHits\n";
#-----------------------------

This is the output :

Ras GTPase    3
catalase    2
cysteine proteinase    4
Total hits: 9

[kschwab]

I believe that \b seems to already do what you want with dashes, and you should be able to reduce your regex down to just: $annotation =~ /\b$term\b/i

See below.

#!/usr/bin/perl
use strict;
use warnings;
my @words=qw(one);
my $text=<<END;
This is one (will match)
This is onething (won't match)
This is one-thing. (will match)
This is a thing-one (will match)
This is a -one (will match)
This is a _one (won't match)
END
foreach my $line (split(/\n/,$text)) {
   foreach my $word (@words) {
       if ($line =~ /\b$word\b/) {
          print "matched [$line]\n";
       } else {
          print "not matched [$line]\n";
       }
   }
}

[mastarr]

Oh great! Thank you very much! That was a very clear and informative little script!

[jwkrahn]

        if($annotation =~ /(\b($term)\b|$term-)\b|(-$term)\b/i) {
            $nTermHits {$term}++;
            $nTotalHits++;
            push(@annotationlist, "$annotation\n");
    }

You are not using the contents of the capturing parentheses so you could simply write that as:

        if($annotation =~ /(?:\b$term|$term-|-$term)\b/i) {





foreach $term (@terms) {
    if ($nTermHits {$term} > 0) {
        printf "%s\t%d\n", $term, $nTermHits {$term};
    }
}

That would be better as:

foreach $term (keys %nTermHits) {
    printf "%s\t%d\n", $term, $nTermHits {$term};
}

[mastarr]

Thanks. That was exactly what I was looking for.

[space_monk]

Alternative method - Read the file in as one slurp, and count the number of occurrences of each term.

Something like this:

# read terms file
open (SRC, $searchTermsFile) || die "can't open search terms file";
@terms = <SRC>;
chomp (@terms);
# read annotations file...
open(my $f, '<', $file) or die "OPENING $file: $!\n";
my $string = do { local($/); <$f> };
close($f);

my %nTermHits;

foreach my $term (@terms) {
   $nTermHits{$term} = () = $string =~ /\b$term\b/ig;
}

I don't have PERL on this machine so any minor syntax issues are your problem :o)

[mastarr]

Thanks for the new method! I tried out your suggestion and it worked no problem.