http://www.perlmonks.org?node_id=1043856


in reply to Re^6: Efficient matching with accompanying data
in thread Efficient matching with accompanying data

it seems that trie optimization brakes for more than 10000 alternatives.

Seems the goalposts have changed slightly since it was first implemented, it was between 12,000 and 13,000 back then .

For most of my stuff, that limit is way to low to consider.

It'd be nice if you'd include the results of your benchmark in your post.

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
  • Comment on Re^7: Efficient matching with accompanying data

Replies are listed 'Best First'.
Re^8: Efficient matching with accompanying data
by LanX (Saint) on Jul 12, 2013 at 02:17 UTC
    actually it breaks around 15000 for me, I think it depends on the total length of the regex-string.

    15000 alternatives: 

    lanx@nc10-ubuntu:~/B/PL/PM/buk_trie$ perl buk_trie.pl chapter2.txt 
481Finding 1027 words (of 5038) took 0.041860 seconds using a hash
482Finding 923 words took 0.025128 seconds using a trie(via regex engi
+ne)

    [download]

    15500 alternatives 

    lanx@nc10-ubuntu:~/B/PL/PM/buk_trie$ perl buk_trie.pl chapter2.txt 
481Finding 1078 words (of 5038) took 0.040339 seconds using a hash
482Finding 958 words took 10.042522 seconds using a trie(via regex eng
+ine)

    [download]

    changing buffer-size didn't help.

    This is perl, v5.10.0 built for i486-linux-gnu-thread-multi

    [download]

    > For most of my stuff, that limit is way to low to consider.

    well as already stated by AnoMonk, you can split the set into different regexes with < 10000 alternatives.

    Regexes can do things which hashes can't...

    Cheers Rolf

    ( addicted to the Perl Programming Language)

[reply]
[d/l]
[select]

      Thanks for posting the stats.

      Regexes can do things which hashes can't...

      Such as .. :)

      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

In Section Seekers of Perl Wisdom