http://www.perlmonks.org?node_id=1208866


in reply to String searching in file

I think that the best way is to load your file into a hash (each IP as a key, the hash value doesn't matter), close the file; then add each IP in your array to the hash (a hash will remoce duplicates anyway); once you've finished that, just write the hash keys (i.e. unique IP addresses) to a file (or to the same file if you're confident that it works OK).

A quick untested example:

open my $IN, "<", $awsLists or die "could not open $awsLists $!"; my %hash = map { chomp; $_ => 1 } <$IN>; close $IN; chomp @command; # just in case you need it. Note: you could choose a b +etter name for your array $hash{$_} = 1 for @command; open my $OUT, ">", "$awsLists.out" or die "could not open $awsLists.ou +t $!"; print "$_\n" for keys %hash; close $OUT;
Using a hash lookup will be usually be much faster than greping your file or its content many times.

Replies are listed 'Best First'.
Re^2: String searching in file
by cbtshare (Monk) on Feb 09, 2018 at 23:04 UTC
    thanks, will test it out.