An optimization of last resort: eliminate capturing from your regexps

[diotalevi]

Once while working on a system that was reading gigs of data and running regexps against it in large chunks. I discovered that a significant part of the time of my runtime was going to running my regular expressions. They weren't overly complex and weren't as good as I'd make them if I were doing the same thing today. I was matching against a string that was around a megabyte and doing it a few million times.

Two improvements really helped out. Getting a simpler, smarter regexp and removing all my capturing. It turns out that even with a dirt simple regexp with almost no backtracking I still wasn't fast enough. It's when I removed all my capturing that I finally hit paydirt. It turns out that perl's regexp engine makes a memcpy (or something similar) of the string so that $1 and similar variables can continue to work even if the source is altered or thrown away.

$str = 'Hulk hate classless system with means of manufacturing given t
+o working class! Hulk crush puny Marxists!';
if ( $str =~ /Hulk hate (\w+)/ ) {
    $str = '';
    my $word = $1;
}

The above snippet shows that for $1 to work the information that was matched has to be fetched from someplace other than $str. That place is some variable internal to the perl interpreter. The penalty is if $str was really large then I've just spent time making really large copies.

I can avoid that penalty if I use offsets. $-[0] contains the offset into the string where the match started and $+[0] contains the offset where the match ended. $1 isn't going to be set so I'll need to use substr() to fetch just the part I need. In the above example I know the match starts at $-[0], skips ten characters then I want whatever is from there on til $+[0].

if ( $str =~ /Hulk hate \w+/ ) {
    # $-[0] == 0
    # $+[0] == 19
    my $word = substr $str, $-[0] + 10, $+[0] - $-[0] + 10;
    # $word is now 'classless'

=pod

[An update. Hue-Bond points out that I should have said $+[0] - $-[0] 
+- 10 instead. Whoops. I just leave this as a note to how easy it is t
+o get this stuff wrong when you do it by hand. This is the kind of er
+ror you could find yourself in when you try this optimization. You we
+re warned]

=cut

}

The optimization saved me real time. It made my process at least an hour or so faster. It went from uh... 3ish hours down to a little over an hour. The code will be difficult to maintain if you do this. Kids, don't make your source code crappy like I did unless you find you really need to. One of your primary goals as a programmer is to reduce unnecessary complexity and if you use this technique it had better be worth it.

[demerphq]

I wonder... A switch could be added that would tell perl that you commit to not modifying the string before utilizing $1. Then the memcopy would be unnecessary. That way you could say

$str =~ /Hulk hate (\w+)/k;

And get the same effect. Of course the demons that would fly out of your nose if you said:

if ($str =~ /Hulk hate (\w+)/k) {
   $str='demons!';
   print $1; # could and probably would throw fatal error.
}

Would be your problem...

BTW, if it isnt clear, this is the reason the memcpy is needed.

Alternatively, perhaps magic could be introduced to $str so that the memcpy would only happen if $str was modified while the match vars pointed at it...

A last possibility would be to not do the memcpy if the string was RO. Then you could by hand readonly the string, do the match, use $1 and then when and if you needed to modify the string undo the RO.

I guess another variant could be that the /k would result in no copy, and an understanding that accessing $1 et all would be a fatal error while that regex was the last used. However the @- @+ arrays would be populated. Its just the user would be expected to do the substring operations by hand.

[tilly]

Making that a flag on Perl is a horrible idea. Quick, I want to speed up my program, can I use that flag? I dunno, do you want to go through all of the modules you import and didn't write?

Make that a flag on the regular expression instead. Then people can turn it on when they find a critical part of their code to optimize. And don't have to worry about looking through modules written by other people.

As tye pointed out, I misread. Sorry.

[tye]

Am I misunderstanding you or did you just not notice the second paragraph:

$str =~ /Hulk hate (\w+)/k;

Note that demerphq has added /k to his regex. I don't see demerphq proposing any per-script or per-invocation flag that you seem to have imagined.

[Aristotle]

If a pattern doesn’t contain (?{}) or (??{}) bits, then $str cannot change during a match. So in that case it would be feasible to postpone the memcpy until right after the match (before the regex engine returns) and memcpy only the matched bits. That way, all regexen which don’t run Perl code would automatically avoid unnecessary copying. I think that would be a worthwhile patch.

[diotalevi]

I think that's what it does already.

[Hue-Bond]

my $word = substr $str, $-[0] + 10, $+[0] - $-[0] + 10;

There seems to be a typo there:

my $str = 'Hulk hate classless system with means of manufacturing give
+n to working class! Hulk crush puny Marxists!';
if ($str =~ /Hulk hate \w+/) {
    print "\$-[0] <$-[0]> \$+[0] <$+[0]>\n";
    my $word = substr $str, $-[0] + 10, $+[0] - $-[0] + 10;
    print "word <$word>\n";
}
__END__
$-[0] <0> $+[0] <19>
word <classless system with means o>

That +10 should be -10.

Anyway, the reason I'm replying is not this, but to suggest some typeglob aliasing to make the code a tiny bit easier to read:

*starts = \@-;
*ends   = \@+;
our (@starts, @ends);

my $str = 'Hulk hate classless system with means of manufacturing give
+n to working class! Hulk crush puny Marxists!';
if ($str =~ /with means of \w+/) {
    print "\$-[0] <$-[0]> \$+[0] <$+[0]>\n";
    my $word = substr $str, $starts[0] + 14, $ends[0] - $starts[0] - 1
+4;
    print "word <$word>\n";
}
__END__
$-[0] <27> $+[0] <54>
word <manufacturing>

[diotalevi]

Hey, that's nicer to look at. Good names are always a plus. I also updated my original post to include your bugfix. I left it there but put a note next to it going "Hey! See! You could screw up like this too if you do this."

[graff]

So, is it really the case that using @- and @+ like you do here does not cause the runtime penalty that is imposed by using the "match" variables ( $`, $&, $') ? If so, that's good to know...

perlvar (in my current 5.8.6) still has the warning about the impact of using any of those three match variables in a script, but that impact is not mentioned at all in the sections about @+ and @- (however the latter does indicate how to use these arrays to get the same values as what the match variables would give you).

[demerphq]

Just so you know... $1 et all are basically ties that do something like

substr($something, $-[$digit], $+[$digit] - $-[$digit]);

Part of the reason for this is that internally perl doesnt populate $1 et all with copies when it runs the regex as when backtracking it could have to recopy the contents of these vars many many times. Wheras updating the appropriate offsets is cheap and constant time.

[diotalevi]

They're not actually ties though. Ties are slow, these aren't.

[davido]

Correct, really, @- and @+ do not impose the performance penalties incurred with $', $&, and $`. Think of the @- and @+ variables as simply keeping track of where something was seen, whereas the 'match' variables keep copies of what was seen. It's quicker to bookmark page nine than it is to memorize it. ;) (ok, that's a bit of a simplistic exaggeration, but it should get the idea across)

[tye]

Hopefully to be somewhat clearer, $`, $&, and $' are especially bad because once they've been seen by Perl, all regexes will do extra work.

Capturing in a regex imparts a performance hit because it means that a copy will be made of the string that the regex is being applied to (which makes it a worse performance hit when matching against really large strings -- one of the worst cases being running a lot of little regexes with capturing against the same huge string, something a parser is likely to do). The performance hit of capturing only applies to the regex that does capturing and most of the time it isn't enough that you'd notice.

@- and @+ are always set by regexes and whether you use them or not doesn't have any performance impact at all.

I'm curious why the regex engine doesn't just copy out the parts that were captured and only copy them after the regex has finished. That seems like a "best of both worlds" solution. Though, I think /k would be really cool, especially if you could use it with:

@matches= $string =~ /$regex/gk;

to make it so it doesn't matter whether $regex contained capturing parens or not. (:

Finally, this benchmark shows that the per-regex performance hit from $`, $&, and $' is the same hit from capturing parens:

#!/usr/bin/perl -w
use strict;
use Benchmark qw( timethis cmpthese );

my %t;
my $pref= "no&";
my $str= "match" . ( "garbage" x 500 );
for( 0, 1 ) {
    $t{$pref."?:"}= timethis( -3, sub {
        $str =~ /(?:match)+/;
    } );
    $t{$pref."()"}= timethis( -3, sub {
        $str =~ /(match)+/;
    } );
    eval '$&';
    $pref= "amp";
}
cmpthese( \%t );
__END__
           Rate amp() no&() amp?: no&?:
amp()  439813/s    --   -0%   -1%  -81%
no&()  441612/s    0%    --   -0%  -81%
amp?:  443563/s    1%    0%    --  -81%
no&?: 2354611/s  435%  433%  431%    --

But, remember: $`, $&, and $' are evil while capturing parens are usually the best solution when you need them and usually don't cause a noticeable performance penalty.

[diotalevi]

True. Using @- and @+ does not invoke the $`, $&, $' penalty. The $@ penalty is just the memcpy penaly I mentioned except it is enabled for all regexps, even the non-capturing ones. Any regexps that already use capturing can't be penalized anymore and aren't any slower because of it.