http://www.perlmonks.org?node_id=181801


in reply to Re: to avoid redundacy in a file
in thread to avoid redundacy in a file

This won't do exactly as the AM wants - some lines will be duplicate to the user, but not to Perl:
$ more file.txt N AB TX NC AB N TX NC FOO BAR N AB TX NC $ perl test.pl file.txt N AB TX NC AB N TX NC FOO BAR

The first two lines of the file.txt file are "the same" to the user, but not to your program. zejames' solution works to the AM's needs, as it creates an unique key for the hash, based on the AM's definition of a duplicate.

Jason