Re: Fast/efficient way of check if a string in an strArr is contained in a line

[betterworld]

if($line=~/$_/i)

On the assumption that you don't actually want to do regex searches I suggest the following:

You can create an index (hash) of the words of those lines that you want to search for. That way you only have to do a hash lookup for each word in the log. I've created a small sample script that demonstrates this.

use strict;
use warnings;

my @logfile = (
    'Aliens ate my baby-sitter',
    'Pearls of Light',
    'Really long line of logs',
);
my @searchfile = (
    'test these words please',
    'ate my',
);

my %wordhash;
for my $line (@searchfile) {
    $line = lc $line;
    # Index the searchlines by their words
    for my $word (split ' ', $line) {
        push @{$wordhash{$word}}, \$line;
    }
}

LOGLINE: for my $line (@logfile) {
    my $lower = lc $line;

    # For every word in the logline, check if we
    # have a searchline containing that word.

    for my $word (split ' ', $lower) {
        for my $searchline (@{$wordhash{$word} || []}) {

            # If the word was found, compare the lines

            if (index($lower, $$searchline) >= 0) {
                next LOGLINE;
            }
        }
    }

    # If there was no matching searchline, print the logline.

    print "Processing the log line $line\n";
}