Re: Junk NOT words

[Anonymous Monk]

How about you index the dictionary file and then work you're way through the string character by character matching against the word. When the next letter results in no further branches in the index it takes that as a word, if the next word results in a dead end try the previous word again minus one character.
Example:
(excuse me for not actually checking against a dictionary for obsucure words)
w>h>e>r>e>r
r=dead end
"where" removed
a>n>g>e>l>s>a
a=dead end
"angels" removed
a>r>e>a>l
l = dead end
"area" removed
l>l>t>h
not making sense, backup
a>r>e>|
stop at "a"
"are" removed
a>l>l>t
t=dead end
"all" removed
ugh... hope you get the idea.

[Anonymous Monk]

The problem with this algorithm, is that it has a VERY bad worse case performance, it's in O(2^n), where n is the length of the string. Meaning that as the strings get larger, this problem will become insolvable by deterministic methods. Some sort of heuristic is needed.