Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

Re: Compiling Regular Expressions

by Cody Pendant (Prior)
on Feb 19, 2003 at 06:42 UTC ( #236562=note: print w/ replies, xml ) Need Help??


in reply to Compiling Regular Expressions

This is kind of the opposite of what the poster is asking for, but what does the function study() actually do?

I say it's the opposite, because the FAQ says it takes more time, and that it works on the scalar to be matched rather than the regex, but I might as well ask.

In what circumstances would I study() a string before hitting it with a regex?
--

“Every bit of code is either naturally related to the problem at hand, or else it's an accidental side effect of the fact that you happened to solve the problem using a digital computer.”
M-J D


Comment on Re: Compiling Regular Expressions
Re: Re: Compiling Regular Expressions
by edan (Curate) on Feb 19, 2003 at 09:22 UTC

    Cody Pendant,

    A few comments... (Monks, correct me if I'm right)

    You are correct that study is the opposite of what the OP wants, but only in your second point, in that it works on the scalar to be matched, and not on the regex itself. I have a feeling that your first point - that it 'takes more time' - is a mis-interpretation of the documentation: study takes extra time up-front to study the scalar in question, if you want to do lots of different regex's on it. Depending on your scalar and your regex's, peforming the study may save time on the regex's because you've already analyzed the heck out of that scalar.

    A possible scenario when this is useful: Let's say your scalar is a paragraph of text. You plan to run all sorts of matches on this text, to get some sort of statistics on it ('foo' was mentioned twice, 'bar' mentioned once, in the word 'barfly', 'um' appeared 3 times, always followed by '...', etc.). Thus you want to study() the same paragraph, to (possibly) save time on all those different regex's you're going to throw at it...

    This is the opposite of what the OP wants, since s/he wants to use the same regex on many different scalars... In this case, I think the previous comments are right on - the regex will be compiled only once if it contains no variable interpolation (i.e. /pattern/), so there's no cause for concern, and using the '/o' operator will force the pattern to be compiled only once if it is in a variable (i.e. /$pattern/o), with the caveat that you cannot change the value of $pattern (well, of course you can, but perl won't notice, and will continue to perform the regex with the value of $pattern when the '/o' was used...)

    Hope this helps (and is accurate)...

      I think your interpretation is correct, though when I've benchmarked it, now and previously, I have had variable results as you'll see below.

      It seems to consistantly slow the regex down (even excluding the study time), if you only match against a single regex.

      There can be some considerable speed up when matching against multiple regexes, but it's not consistant in how much you get. In the example below, matching against 2 regexes on a study'd string sometimes shows upto 50% improvement relative to an unstudy'd one, but less so when matching 3 regexes against the same two strings. It seems to be a function of the constant content of the regexs. Ie. what characters are involved. Those containing rare characters showing greater benefit than those not, but that's really educated speculation.

      That said, there always seems to be some benefit to studying the string if you intend to match against more than once with 2 or more different regexes.

      There doesn't seem to be any benefit (actually normally a small penalty) for multiple matches with a single regex, as with the /g modifier.

      perl> use Benchmark qw[cmpthese]; perl> $s = "I say it's the opposite, because the FAQ says it takes mo +re time, and that it works on the scalar to be matched rather than the regex, but I might as well ask. In what circumstances would I + study() a string before hitting it with a regex?" perl> $t = $s perl> study $s perl> cmpthese( -1, { studied=>'$_=()=$s=~m[FAQ]o', slacker=>'$_=()=$ +t=~m[FAQ]o' } ) Benchmark: running slacker, studied, each for at least 1 CPU seconds. +.. slacker: 1 wallclock secs ( 1.01 usr + 0.00 sys = 1.01 CPU) @ 56 +776.24/s (n=57344) studied: 2 wallclock secs ( 1.12 usr + 0.00 sys = 1.12 CPU) @ 54 +807.31/s (n=61439) Rate studied slacker studied 54807/s -- -3% slacker 56776/s 4% -- perl> cmpthese( -1,{\ studied=>'$a = ($s = ~m[FAQ]o && $s =~ m[regex]o)',\ slacker=>'$b = ($t =~ m[FAQ]o && $t =~ m[regex]o)',\ }) Benchmark: running slacker, studied, each for at least 1 CPU seconds. +.. slacker: 2 wallclock secs ( 1.08 usr + 0.00 sys = 1.08 CPU) @ 98 +754.16/s (n=106852) studied: 2 wallclock secs ( 1.20 usr + 0.00 sys = 1.20 CPU) @ 65 +054.91/s (n=78196) Rate studied slacker studied 65055/s -- -34% slacker 98754/s 52% -- perl> cmpthese( -1,{\ studied=>'$a = ($s =~ m[FAQ]o && $s =~ m[regex]o) && $s =~ m[circumsta +nces]o',\ slacker=>'$b = ($t =~ m[FAQ]o && $t =~ m[regex]o) && $t =~ m[circumsta +nces]o',\ }) Benchmark: running slacker, studied, each for at least 1 CPU seconds. +.. slacker: 2 wallclock secs ( 1.00 usr + 0.00 sys = 1.00 CPU) @ 71 +679.00/s (n=71679) studied: 2 wallclock secs ( 1.17 usr + 0.00 sys = 1.17 CPU) @ 56 +503.84/s (n=66166) Rate studied slacker studied 56504/s -- -21% slacker 71679/s 27% -- perl> cmpthese( -10,{\ studied=>'$a = ($s =~ m[FAQ]o && $s =~ m[regex]o)',\ slacker=>'$b = ($t =~ m[FAQ]o && $t =~ m[regex]o)',\ }) Benchmark: running slacker, studied, each for at least 10 CPU second +s... slacker: 10 wallclock secs (11.21 usr + 0.01 sys = 11.22 CPU) @ 1 +11296.69/s (n=1248415) studied: 11 wallclock secs (10.00 usr + 0.00 sys = 10.00 CPU) @ 1 +41637.98/s (n=1417088) Rate slacker studied slacker 111297/s -- -21% studied 141638/s 27% --

      Examine what is said, not who speaks.

      The 7th Rule of perl club is -- pearl clubs are easily damaged. Use a diamond club instead.

        A reading of _Mastering Regular Expressions_ leaves me with the impression that sometimes study() speeds things up but other times it just slows things down. So its actual behaviour on performance is likely dependant on your data, your regexes (which is also another sort of data) and then how often each data+regex combination is used. The way M_R_E leaves things, I'm not even sure you can arrive at any sort of truth by using benchmarks. Or... you can arrive at a local definition of truth only by testing your specific data + your specific regexes together.

        The general idea is that a generalized benchmark for study() is invalid.


        Seeking Green geeks in Minnesota

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://236562]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (13)
As of 2014-12-18 20:13 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (61 votes), past polls