|more useful options|
How to remove duplicates from a large set of keysby nite_man (Deacon)
|on Feb 10, 2005 at 08:06 UTC||Need Help??|
nite_man has asked for the wisdom of the Perl Monks concerning the following question:
I have a problem with finding dublicated values in the big set of keys. So, let's imagine. I have the set of values. It's about 1 million. In real time I should check does a new value exist in my set and if not to add it.
One way is to use a hash. But it takes a lot of time when number of values is increased.
Other way is store values into a database and make unique index by values. So, when a new value is coming I can try to insert it into a table and catch exceptions.
I'd like to hear your opinions and suggestions about it.TIA
It's only my opinion and it doesn't have pretensions of absoluteness!
Retitled by davido from 'Finding duplicated values'.