Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re: Remove Duplicates!! Please Help

by jbert (Priest)
on Jan 03, 2008 at 15:42 UTC ( #660224=note: print w/ replies, xml ) Need Help??


in reply to Remove Duplicates!! Please Help

If the lines are adjacent the unix tool 'uniq' does this job:

uniq input_file > output_file
If they're not adjacent, you can sort the file first (unless, as the line endings suggest, there is other structure to the file such as an HTML or XML header). This is so useful that sort has it as an option (-u), so you don't need to pipe to uniq:
sort -u input_file > output_file.


Comment on Re: Remove Duplicates!! Please Help
Select or Download Code
Re^2: Remove Duplicates!! Please Help
by davidrw (Prior) on Jan 03, 2008 at 15:58 UTC
    While i love uniq, it's not a solution here. OP said "I want to remove duplicate entries for one single instance of COMPUTER column." Not eliminate duplicate lines, which your uniq examples doe.

    My first action after reading OP was to man uniq -- there's options to "avoid comparing the first N fields" and "avoid comparing the first N characters", but unfortunately neither of those work for comparing just the first column (of a tab-delim'd file).
      Fair point, sorry. I misread the sample data and gave an alternative to the (also incorrect) perl version above. Thanks for the catch.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://660224]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (15)
As of 2014-07-11 20:58 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    When choosing user names for websites, I prefer to use:








    Results (235 votes), past polls