Beefy Boxes and Bandwidth Generously Provided by pair Networks
good chemistry is complicated,
and a little bit messy -LW

Re: Remove duplicate data from text file

by CharlesClarkson (Curate)
on Nov 28, 2001 at 09:02 UTC ( #128002=note: print w/replies, xml ) Need Help??

in reply to Remove duplicate data from text file

If you need to maintain the original order and delete subsequent duplicates, you could try this.

{ my %seen; my ($in_file, $out_file) = qw| in.txt out.txt|; open my $in_fh, $in_file or die "$in_file: $!"; open my $out_fh, '>', $out_file or die "$in_file: $!"; while ( <$in_fh> ) { print $out_fh $_ unless $seen{$_}++; } }

Charles K. Clarkson

Why isn't phonetic spelled the way it sounds?

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://128002]
[Corion]: ... an aggregator site that also listed (ideally) all of these links once. And maybe also have an exclude list to blacklist some of the most spammy links that the site may have never mentioned at all
[marto]: I gave up reading HN at all, and feel much better for it :P
[Corion]: Now, how to best automate that Google search ... :-)
[Corion]: marto: Yeah, I'm also short of not reading it at all. I already stopped reading it in the morning because it gave me a foul mood.
[Corion]: The good posts on HN are the non-computing posts. Most of the other stuff is maybe relevant to you if you are 20 and live in Silicon Valley...
[Corion]: Two attributes that don't describe me.
[Corion]: Maybe I should also write a curator for HN or simply not read it anymore, like you do.

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (7)
As of 2017-07-24 13:26 GMT
Find Nodes?
    Voting Booth?
    I came, I saw, I ...

    Results (354 votes). Check out past polls.