Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Re: Remove duplicate data from text file

by CharlesClarkson (Curate)
on Nov 28, 2001 at 09:02 UTC ( #128002=note: print w/ replies, xml ) Need Help??


in reply to Remove duplicate data from text file

If you need to maintain the original order and delete subsequent duplicates, you could try this.

{ my %seen; my ($in_file, $out_file) = qw| in.txt out.txt|; open my $in_fh, $in_file or die "$in_file: $!"; open my $out_fh, '>', $out_file or die "$in_file: $!"; while ( <$in_fh> ) { print $out_fh $_ unless $seen{$_}++; } }



HTH,
Charles K. Clarkson


Why isn't phonetic spelled the way it sounds?


Comment on Re: Remove duplicate data from text file
Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://128002]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others meditating upon the Monastery: (6)
As of 2015-07-04 21:30 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (60 votes), past polls