Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Re: Remove duplicate data from text file

by CharlesClarkson (Curate)
on Nov 28, 2001 at 09:02 UTC ( #128002=note: print w/ replies, xml ) Need Help??


in reply to Remove duplicate data from text file

If you need to maintain the original order and delete subsequent duplicates, you could try this.

{ my %seen; my ($in_file, $out_file) = qw| in.txt out.txt|; open my $in_fh, $in_file or die "$in_file: $!"; open my $out_fh, '>', $out_file or die "$in_file: $!"; while ( <$in_fh> ) { print $out_fh $_ unless $seen{$_}++; } }



HTH,
Charles K. Clarkson


Why isn't phonetic spelled the way it sounds?


Comment on Re: Remove duplicate data from text file
Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://128002]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (22)
As of 2014-12-18 18:47 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (59 votes), past polls