Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?

Re: Remove Duplicate Lines

by tstock (Curate)
on Sep 18, 2002 at 19:01 UTC ( #198965=note: print w/replies, xml ) Need Help??

in reply to Remove Duplicate Lines

If this is a one time fix, you can use the unix commands 'sort' and 'uniq', like so:
sort db.txt | uniq > newfile.db
'sort' sorts the entries in db.txt, uniq then cuts out duplicates and the the '>' redirects the output to a new file name instead of the screen (STDOUT).