Re: Strange behaviour of the regex engine (pathological protection)

[tye]

Perl optimizes certain nested loop regex constructs similar to (...*)* in order to avoid what is pathological behavior with most regex engines. For non-pathological cases, there is a small cost to this protection.

The reason the performance significantly decreases is because that crosses the point that triggers the warning "Complex regular subexpression recursion limit (32766) exceeded" (repeatedly).

[thof]

Many thanks! I didn't know about warnings. But I noticed new issue. The problem is that when this warning appears the regex doesn't match... I changed line to print "not matched" if $TestString !~ m/^(a+|b)*$/; and for string 256000 and above regex is not able to match. Also when I use possessive quantifiers (m/^(a++|b)*+$/) performance improves a lot but still it does not match for strings larger than 256000.

I don't understand regex engine in Perl... I also checked the same regexes for Python and PHP (PCRE) and they work similar to Java. I doubt that this is bug in Perl and I cannot match big input strings. There must be something I don't know.

[tye]

If I were you, I would file this (the failure case) as a bug against Perl. If nothing else, it might lead to p5p giving you better information than I have off the top of my head.

Best case, this 32k limit might be removed (a limit that seemed more reasonable when it was implemented but seems increasingly small as time passes).

[Anonymous Monk]

Many thanks! I didn't know about warnings. But I noticed new issue. The problem is that when this warning appears the regex doesn't match...

It's not a new issue it's the same issue :) The recursion depth is limited to prevent catastrophic backtracking (mkay I think it's not actually recursion but it's called that in the docs). Doesn't Friedl describe that in his book? Add use diagnostics; to your program.