Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

Help to improve script that searches a file for key words in a second file.

by mastarr (Novice)
on Mar 29, 2013 at 16:14 UTC ( #1026165=perlquestion: print w/ replies, xml ) Need Help??
mastarr has asked for the wisdom of the Perl Monks concerning the following question:

Hello everybody,

I'm searching a file of annotations for a select list of terms in a second file. It seems to work fine but I'd like to learn alternative ways to do it or learn how to improve it. Specifically, is there is a way I can improve the regular expressions in this line:

 /(\b($term)\b|$term-)\b|(-$term)\b/i) {

Thanks!

Here is the full script:

#!/usr/bin/perl # Read search terms into array @terms open (SRC, $searchTermsFile) || die "can't open search terms file"; @terms = <SRC>; chomp (@terms); @terms = sort @terms; my @annotationlist; #------------------------------------------------------- # Search each line in annotations file for match with search terms %nTermHits = (); $nTotalHits = 0; open (ANN, $annotationsFile) || die "$annotationsFile won't open\n"; while ($line = <ANN>) { #chomp($line); @fields = split(/\t/, $line); $annotation = $fields [2]; foreach $term (@terms) { if($annotation =~ /(\b($term)\b|$term-)\b|(-$term)\b/i) { $nTermHits {$term}++; $nTotalHits++; push(@annotationlist, "$annotation\n"); } }} #@annotationlist = sort @annotationlist; #print "@annotationlist\n"; #--------------------------------------- foreach $term (@terms) { if ($nTermHits {$term} > 0) { printf "%s\t%d\n", $term, $nTermHits {$term}; } } print "Total hits: $nTotalHits\n"; #-----------------------------

This is the output :

Ras GTPase 3 catalase 2 cysteine proteinase 4 Total hits: 9

Comment on Help to improve script that searches a file for key words in a second file.
Select or Download Code
Re: Help to improve script that searches a file for key words in a second file.
by kschwab (Priest) on Mar 29, 2013 at 16:45 UTC
    I believe that \b seems to already do what you want with dashes, and you should be able to reduce your regex down to just: $annotation =~ /\b$term\b/i

    See below.
    #!/usr/bin/perl use strict; use warnings; my @words=qw(one); my $text=<<END; This is one (will match) This is onething (won't match) This is one-thing. (will match) This is a thing-one (will match) This is a -one (will match) This is a _one (won't match) END foreach my $line (split(/\n/,$text)) { foreach my $word (@words) { if ($line =~ /\b$word\b/) { print "matched [$line]\n"; } else { print "not matched [$line]\n"; } } }

      Oh great! Thank you very much! That was a very clear and informative little script!

Re: Help to improve script that searches a file for key words in a second file.
by jwkrahn (Monsignor) on Mar 29, 2013 at 16:51 UTC
    if($annotation =~ /(\b($term)\b|$term-)\b|(-$term)\b/i) { $nTermHits {$term}++; $nTotalHits++; push(@annotationlist, "$annotation\n"); }

    You are not using the contents of the capturing parentheses so you could simply write that as:

    if($annotation =~ /(?:\b$term|$term-|-$term)\b/i) {


    foreach $term (@terms) { if ($nTermHits {$term} > 0) { printf "%s\t%d\n", $term, $nTermHits {$term}; } }

    That would be better as:

    foreach $term (keys %nTermHits) { printf "%s\t%d\n", $term, $nTermHits {$term}; }

      Thanks. That was exactly what I was looking for.

Re: Help to improve script that searches a file for key words in a second file.
by space_monk (Chaplain) on Mar 29, 2013 at 17:13 UTC
    Alternative method - Read the file in as one slurp, and count the number of occurrences of each term.

    Something like this:

    # read terms file open (SRC, $searchTermsFile) || die "can't open search terms file"; @terms = <SRC>; chomp (@terms); # read annotations file... open(my $f, '<', $file) or die "OPENING $file: $!\n"; my $string = do { local($/); <$f> }; close($f); my %nTermHits; foreach my $term (@terms) { $nTermHits{$term} = () = $string =~ /\b$term\b/ig; }
    I don't have PERL on this machine so any minor syntax issues are your problem :o)
    A Monk aims to give answers to those who have none, and to learn from those who know more.

      Thanks for the new method! I tried out your suggestion and it worked no problem.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1026165]
Approved by toolic
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (5)
As of 2014-12-20 07:44 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (95 votes), past polls