Re: Finding Nearly Identical Sets

[herveus]

Howdy!

Using the word "set" is a bit confusing. What I think you have is a string of digits. Going from that, I think some sort of Levenshtein distance is involved here. I note the existance of Text::Levenshtein on CPAN.

[Limbic~Region]

herveus,

It isn't a string of digits though you could think of them that way.

Have you ever tried executing the Levenshtein edit distance a trillion times? Even the XS version isn't that fast. Let's say I get 2 million messages a day and I have 500K different sets/strings to compare against - this isn't the way to go.