Pathologically Eclectic Rubbish Lister | |
PerlMonks |
Re^6: Efficient matching with accompanying databy LanX (Saint) |
on Jul 12, 2013 at 01:40 UTC ( [id://1043853]=note: print w/replies, xml ) | Need Help?? |
Unfortunately I've spend some time I don't have into this! :( As it seems that trie optimization brakes for more than 15000 alternative patterns. up to this number the performance is competitive:
please note that I somewhere when experimenting I introduced a bug which skipped 100 matches... couldn't find the whole of Darwin's text and took chapter2, and as a dictionary I took (and cleaned) the en-US.dic in mozillas path. Please note some changes I did: 1. I sorted descending by size not ascending, to fulfill the OP's requirements of multiword matches. 2. You didn't use word delimiter, such that the regex has to start matching in the middle of the word. Thats why I put spaces around the parens. 3. I don't know why you lowercased the input, my dict is case-sensitive, as a side note perl has lc 4. I don't see the point of looping over m//g if you can get all matches in one attempt. 5. I precompiled the regex to avoid recompilations. > I'd love to see a demonstration of the trie optimisation actually doing something. OK, I'm posting the code now as it is as a starting point for everyone who wants to experiment with it: Disabling the buffer (negative value) will show the effect of missing trie. have fun!
Cheers Rolf ( addicted to the Perl Programming Language) PS: The requirements of the OP weren't clear for me, if only whole words are of interest I recommend sticking with the hash method.
In Section
Seekers of Perl Wisdom
|
|