http://www.perlmonks.org?node_id=660217


in reply to Remove Duplicates!! Please Help

A one-liner:

perl -lnae 'print unless $seen{$F[0]}++' < file2.txt > file1.txt

Update: The OP needed the last entry if there are duplicates, the above solution took the first entry. Here's an alternative solutionthat requires module Tie::Hash::Indexed to be installed:

$ perl -MTie::Hash::Indexed -lane '\ sub BEGIN {tie %seen, "Tie::Hash::Indexed"};\ sub END {print $seen{$_} for keys %seen};\ $seen{$F[0]} = $_' file2.txt > file1.txt