Trying to sound knowledgeable without knowing why or how you got the results you did, I would suspect that this is something that you can't rely on in every version of Perl, and possibly even system to system.

Right, forgot that ... I tried it on 5.10, 5.16, and 5.17.6, with almost identical results. As to a system, I really doubt that it matters.

Perhaps the following quote from the Camel Book will shed some light on this question:

Short-circuit alternation is often faster than the corresponding regex. So:

print if /one-hump/ || /two/;
[download]

is likely to be faster than:

print if /one-hump|two/;
[download]

at least for certain values of one-hump and two. This is because the optimizer likes to hoist certain simple matching operations up into higher parts of the syntax tree and do very fast matching with a Boyer-Moore algorithm. A complicated pattern tends to defeat this.
— Tom Christiansen, brian d foy & Larry Wall with Jon Orwant, Programming Perl (4^th Edition, 2012), p. 692.

Hope that helps,

Not really, because it says:

A complicated pattern tends to defeat this.

and i'm seeing exactly the opposite. I wish Tom would comment on that :) But thank you for the quote, it helps with understanding why I think that the observed behavior is bad.

Perhaps read "complicated" as "non-trivial", EG: having alternations

1. "Is this a recognized problem? Yes, see Athanasius', above.
2. "Is it a problem at all? Yes, but not one that's apt to be resolved other than by careful choice among the alternate approaches.
3. "Does it look like it needs to be reported as a bug? No; see 1 above
4. Does this "mean that alterations with grouping should be avoided at all...?" Definitely not; sometimes the difference in speed is too small to make any difference; sometimes the clarity of one approach clearly outweighs any other issues; and sometimes other factors, like personal taste, can be allowed to determine. Just be sure to think carefully about which applies.

It's not clear to me what you are trying to achieve with your regex.

The simple grouping that look like this

runs quickly, it's only the one with lots of capture groups that is slow. i.e

So maybe there's just a better way to get the result you want, if you'd care to explain what that is?

(I work with dk.)

Another question could be: why is the one with the capture groups so slow, since none of the words match the string?

And in general, why is alternation&capture so much slower than looping&capture + alternation combined?

The reason for the code is to replace code with 60 or so similarly structured regexes in a library used by a couple of legacy applications with an automatically generated regex generated with info from configuration files, for both (potential) performance gains, allowing different behaviour across applications, and definite maintainability gains. The strings replaced all have the structure \bFOO:\s*bar(\d+) or \bBAZ:\s*(\w+) etc.

Suggestions like "Well, don't do that" are likely to go unheard :-)

OK then, If you want to use a non-optimal solution for operational reasons, go right ahead :-)

Added to balker's response, it's not that we're trying to achieve, we know other means how to get where we want to, but it's about the principle I've long nourished, (see Anastasius's quote above), and now it doesn't hold water. What i'd love to see, an explanation of someone who knows why regex algorithm exhibits behavior that is CONTRARY to perl lore.

Thank you, but your statement is contradicting itself. We're seeing milliseconds wasted in the regex (for a web-app!), which is why we bothered to examine why in the first place.

my $q1s = join('|', @matchwords);
my $q1  = qr/$q1s\s*\w+/;

my $q3s = join('|', @matchwords);
my $q3  = qr/($q3s)\\s*\\w+/;
[download]

alternation, grouping:  1 wallclock secs ( 0.58 usr +  0.00 sys =  0.5
+8 CPU) @ 1724137.93/s (n=1000000)
alternation, no grouping:  5 wallclock secs ( 4.85 usr +  0.00 sys =  
+4.85 CPU) @ 206185.57/s (n=1000000)
loop, grouping: 24 wallclock secs (23.33 usr +  0.00 sys = 23.33 CPU) 
+@ 42863.27/s (n=1000000)
loop, no grouping: 23 wallclock secs (22.57 usr +  0.00 sys = 22.57 CP
+U) @ 44306.60/s (n=1000000)
[download]

Dave.

But that doesn't leave room for the various match-words to have different "\w+"'s - e.g. /\bFOO:\s*bar(\d+)/ or /\bBAZ:\s*(\w+)/

I guess I'm wondering why the trie-optimization isn't used for fixed string prefixes as well (as I'd assumed before I wrote the code).

Actually I stand corrected: the fixed string prefixes are collected together into a trie where there are (possibly differing) wildcard suffixes; the killer is the individual captures, which disables the trie optimisation.

So, alternation is the fastest, as long as you put any captures outside the alt.

Dave.