Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?

Comparing likeness of two strings

by RiotTown (Scribe)
on Jan 19, 2005 at 20:14 UTC ( #423489=perlquestion: print w/replies, xml ) Need Help??
RiotTown has asked for the wisdom of the Perl Monks concerning the following question:

I'm in the process of putting some anti-fraud tools into place for my company. Part of this is to compare strings from an input form against those already existing within a database (all free form). IE, comparing "Steven Smith" to "Stephen Smithe" and "Steve Smith". This isn't limited to names right now, just makes for easier explaining.

I've got an algorithm and query already set up to return rows that are close to the user-input text, but I'm running into problems displaying exactly how similar the two strings are for ranking purposes.

Currently I'm testing with good results the Levenshtein Distance algorithm. The problem is that this method is SLOW, especially when comparing longer strings. (info can be found here on the Levenshtein Distance algorithm.

Does anyone know of a faster way to get a meaningful representation of how closely two stings match?

Replies are listed 'Best First'.
Re: Comparing likeness of two strings
by waswas-fng (Curate) on Jan 19, 2005 at 20:29 UTC
    Text::LevenshteinXS looks like it may be faster than the pure perl implementation. Also you can get other viewpoints with added info from modules like: Text::WagnerFischer and one of the soundex or metaphone modules on cpan.

Re: Comparing likeness of two strings
by VSarkiss (Monsignor) on Jan 19, 2005 at 20:24 UTC
Re: Comparing likeness of two strings
by Limbic~Region (Chancellor) on Jan 19, 2005 at 20:27 UTC
Re: Comparing likeness of two strings
by RiotTown (Scribe) on Jan 19, 2005 at 22:04 UTC
    I'm going to give the Text::LevenshteinXS module a try to find out if that implementation is really significantly faster than the pure perl function that I've got now.

    Thanks for the input!
      Huge improvement. I've seen increases in performance of 2-3x over the pure Perl. Once again, thanks!

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://423489]
Approved by amw1
[shmem]: well I use I2C and SPI and stuff, but creating a pseudo chip looks to me like lot of indirection and memory clutter... not?
[choroba]: Are you going to use the device soon? Related to your comment about "not having much time to do a lot of coding"...
[stevieb]: sure, but I'm just learning ;) I consider it practice to get a good understanding of what goes on *after* an I2C/SPI request is made
[shmem]: ah ok. Gonna read that. but now....
shmem compiles himself into his template
[stevieb]: this is my RPi:: distribution overall automation build system physical layout. It doesn't encompass all of my RPi dists yet, ...
[stevieb]: ...but I'm getting close. I've been soldering my own PCBs to gain that experience as well, before I have my own manufactured.
[stevieb]: heh, nice *template* ;) 'night shmem!

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (5)
As of 2017-06-25 22:50 GMT
Find Nodes?
    Voting Booth?
    How many monitors do you use while coding?

    Results (572 votes). Check out past polls.