Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?

Re: Re: Junk NOT words

by kshay (Beadle)
on Oct 31, 2002 at 16:25 UTC ( #209427=note: print w/replies, xml ) Need Help??

in reply to Re: Junk NOT words
in thread Junk NOT words

I wonder if you' d get better results by building a Markov chain based not on the frequency with which a letter follows another individual letter, but on the frequency with which a letter follows a given pair of letters.

I recently did something like this for a web application that goes the other direction: instead of trying to recognize letter combinations as "words," it generates "words" that "make sense" phonetically based on a given data set—in this case, lists of names. You can play with it here. Try a couple of different categories and you'll see that the resulting made-up names are quite phonetically distinctive; they really do seem French or Shakespearean (and within that, male or female) depending on what category you select.

I got the basic approach from this page by Chris Pound. I can't post the code because I wrote it for my employer, but basically I took each list of names, calculated the frequencies, and generated a Perl library containing a long series of assignments like this:

@{$pairs{'ri'}}{qw(a c e n s u)} = (3, 1, 1, 3, 1, 1);

That means that within this data set, "ri" is followed by "a" three times, "c" one time, etc.

Anyway, I'd be curious how this approach might work for word recognition, and how the results for "pair-letter" frequencies might differ from those obtained with "letter-letter" frequencies.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://209427]
[LanX]: what is the intention of your question?
[LanX]: why do you need to see the nodelet without votes left???
[marinersk]: LanX, the problem is that the lack of interface visual stability is disconcerting to me. The Nodelet appearing and disappearing is distracting, and despite over a decade, still troubles me. It jangles me.
[LanX]: I see
[marinersk]: I appear to be unusually dependent on spacial stability. I have had trouble from someone moving an object a mere four inches to the left.

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (7)
As of 2017-05-29 13:51 GMT
Find Nodes?
    Voting Booth?