Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Re: Search for similar strings - to standardise

by Rival (Acolyte)
on Oct 30, 2009 at 16:15 UTC ( #804191=note: print w/replies, xml ) Need Help??


in reply to Search for similar strings - to standardise

For the string comparison, using the Levenshtein distance is probably your best bet. This is often used in spell-checking to suggest corrections.

For implementations, there is a Text::Levenshtein module on CPAN. But also, take a look at this node and this node.

  • Comment on Re: Search for similar strings - to standardise

Replies are listed 'Best First'.
Re^2: Search for similar strings - to standardise
by educated_foo (Vicar) on Oct 31, 2009 at 03:33 UTC
    Edit distance would be useful for comparison, but not so much for clustering strings with their common misspellings. For that, you might try n-grams (mentioned above) or locality-sensitive hashing (basically the same thing, but with gaps).

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://804191]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (10)
As of 2016-10-01 21:20 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    How many different varieties (color, size, etc) of socks do you have in your sock drawer?






    Results (7 votes). Check out past polls.