Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask
 
PerlMonks  

Re: Files

by arturo (Vicar)
on May 23, 2001 at 19:09 UTC ( [id://82585]=note: print w/replies, xml ) Need Help??


in reply to Removing duplicate lines from files (was 'Files')

most straightforward answer, assuming you have duplicate lines. Whether this is a good idea depends also on the size of the file and your system memory, so keep that in mind.

Basic technique: load the lines into an array, then remove the duplicate members of that array. Easiest way to do that is by mapping the members of the array into the keys of a hash (hash keys cannot be duplicated).

open FILE, $filename or die "Can't open $filename\n"; my @lines = <FILE>; close FILE; my @new_list = keys map {$_ => 1} @lines;

Then just print out @new_list to the new version of the file.

update if you're on a *nix-type system, sort filename | uniq > filename.new will do the same thing, albeit by sorting the lines first. The perl technique won't necessarily preserve the original order, either, but has more general application.

HTH.

perl -e 'print "How sweet does a rose smell? "; chomp ($n = <STDIN>); +$rose = "smells sweet to degree $n"; *other_name = *rose; print "$oth +er_name\n"'

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://82585]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others perusing the Monastery: (6)
As of 2024-03-29 01:19 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found