http://www.perlmonks.org?node_id=198964


in reply to Remove Duplicate Lines

If you are on a Unix system that has a sort command that
supports the -u switch ( -u implies Unique ) you could do this
$ sort -u db.txt > outdb.txt
I know that Solaris's sort has -u and Red Hats cygwin does not.
I hope that helps :)