Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?

Comparing likeness of two strings

by RiotTown (Scribe)
on Jan 19, 2005 at 20:14 UTC ( [id://423489]=perlquestion: print w/replies, xml ) Need Help??

RiotTown has asked for the wisdom of the Perl Monks concerning the following question:

I'm in the process of putting some anti-fraud tools into place for my company. Part of this is to compare strings from an input form against those already existing within a database (all free form). IE, comparing "Steven Smith" to "Stephen Smithe" and "Steve Smith". This isn't limited to names right now, just makes for easier explaining.

I've got an algorithm and query already set up to return rows that are close to the user-input text, but I'm running into problems displaying exactly how similar the two strings are for ranking purposes.

Currently I'm testing with good results the Levenshtein Distance algorithm. The problem is that this method is SLOW, especially when comparing longer strings. (info can be found here on the Levenshtein Distance algorithm.

Does anyone know of a faster way to get a meaningful representation of how closely two stings match?

Replies are listed 'Best First'.
Re: Comparing likeness of two strings
by waswas-fng (Curate) on Jan 19, 2005 at 20:29 UTC
    Text::LevenshteinXS looks like it may be faster than the pure perl implementation. Also you can get other viewpoints with added info from modules like: Text::WagnerFischer and one of the soundex or metaphone modules on cpan.

Re: Comparing likeness of two strings
by VSarkiss (Monsignor) on Jan 19, 2005 at 20:24 UTC
Re: Comparing likeness of two strings
by Limbic~Region (Chancellor) on Jan 19, 2005 at 20:27 UTC
Re: Comparing likeness of two strings
by RiotTown (Scribe) on Jan 19, 2005 at 22:04 UTC
    I'm going to give the Text::LevenshteinXS module a try to find out if that implementation is really significantly faster than the pure perl function that I've got now.

    Thanks for the input!
      Huge improvement. I've seen increases in performance of 2-3x over the pure Perl. Once again, thanks!

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://423489]
Approved by amw1
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others exploiting the Monastery: (4)
As of 2024-06-22 06:44 GMT
Find Nodes?
    Voting Booth?

    No recent polls found

    erzuuli‥ 🛈The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.