|Perl: the Markov chain saw|
OK, I'll take you through it... it looks like I misled several others with my late night post as well.
First, lets look at a slightly newer version of the code:
Our main goal is to construct a pattern that matches a piece at the end of the string. This match needs to be one of the keys in %sufdata, specifically the longest such key that matches.
So, start with line #1.
Here we create a list of keys in %sufdata, then join them together with '|' symbols. $pattern is now a string that looks like "a|s|k|ar|ic|ec|ek|er|vec" though the order of the keys is unknown (and unimportant in this specific case).
In line #2:
If we expand $pattern oursleves, we get:
We check for a match, and replace the first one we find with the corresponding value from %sufdata.
Now here's the trick... the regex engine works from left to right, checking if *any* of those keys match at each position along the way. Since we are anchored at the end, and we use the first one we find, this effectively becomes a search for the longest matching suffix.
So, because of the "left most matching" behavior of the regex engine, we can feel assured that the first match, will actually be the longest one in this particular instance.
Lines 3-6 are left as an excercise for the reader. ;-)