jodaka has asked for the wisdom of the Perl Monks concerning the following question:
Hello there
let's assume we have some string '20090102' which is date in ISO format. And we want to match/parse it against regexp. I wonder what method of writing regexp would be more efficient:
/(\d\d\d\d)(\d\d)(\d\d)/
or
/(\d{4})(\d{2})(\d{2})/
so mainly question in what is better: \d\d\d\d or \d{4} ?
:)
Thanks
Re: better (faster) way of writing regexp
by almut (Canon) on Dec 02, 2009 at 13:11 UTC
|
I'd say use whichever is more readable — that's "better", IMHO. I don't think there is much of a performance difference (if any), but if you really want to know, Benchmark it.
| [reply] |
|
use strict;
use Benchmark;
my $results = timethese( 1e6,
{
repeat => sub{
my $t1 = '20090123';
$t1 =~ /(\d\d\d\d)(\d\d)(\d\d)/;
my ($y1,$m1,$d1) = ($1,$2,$3);
},
range => sub{
my $t2 = '20090123';
$t2 =~ /(\d{4})(\d{2})(\d{2})/;
my ($y2,$m2,$d2) = ($1,$2,$3);
},
} );
my $results2 = timethese( 1e6,
{
range2 => sub{
my $t3 = '20090123';
$t3 =~ /(\d{4})(\d{2})(\d{2})/;
my ($y3,$m3,$d3) = ($1,$2,$3);
},
repeat2 => sub{
my $t4 = '20090123';
$t4 =~ /(\d\d\d\d)(\d\d)(\d\d)/;
my ($y4,$m4,$d4) = ($1,$2,$3);
},
}
);
__END__
Benchmark: timing 1000000 iterations of range, repeat...
range: 2 wallclock secs ( 1.69 usr + 0.00 sys = 1.69 CPU) @ 59
+2417.06/s (n=1000000)
repeat: 2 wallclock secs ( 1.53 usr + 0.00 sys = 1.53 CPU) @ 65
+3167.86/s (n=1000000)
Benchmark: timing 1000000 iterations of range2, repeat2...
range2: 2 wallclock secs ( 1.70 usr + 0.00 sys = 1.70 CPU) @ 58
+6854.46/s (n=1000000)
repeat2: 0 wallclock secs ( 1.53 usr + 0.00 sys = 1.53 CPU) @ 65
+3167.86/s (n=1000000)
| [reply] [d/l] |
|
| [reply] |
|
|
|
Appended qw(:all) to use Benchmark line.
Changed 1e6 to -5 to run for 5 seconds, rather than a particular count ... not that it makes much difference.
Changed timethese to cmpthese to generate the following table.
Rate range repeat
range 483495/s -- -5%
repeat 507038/s 5% --
Rate range2 repeat2
range2 485834/s -- -4%
repeat2 506114/s 4% --
I wouldn't get excited about a 4% or 5% difference ... bet you're making 25% inefficiencies elsewhere, assuming you aren't using algorithmns that are costing you hundreds of %. I don't mean just you, anybody's code, including mine.
--
TTTATCGGTCGTTATATAGATGTTTGCA
| [reply] |
Re: better (faster) way of writing regexp
by moritz (Cardinal) on Dec 02, 2009 at 13:37 UTC
|
As others have said, the speed difference can likely be neglected. But you should ask yourself if \d is really what you want. \d matches all Unicode characters that are digits, including many non-ASCII characters. Can your processing code later on handle those?
If not, use [0-9] instead.
It might be a tad faster to match it all in one capture, and unpack the individual pieces later, but you should really benchmark it before using it.
use strict;
use warnings;
use 5.010;
my $str = '20091224';
if ($str =~ m/([0-9]{8})/) {
say join '|', unpack "A4 A2 A2", $1;
}
But much more than such micro optimizations are gained by anchors: If you know something about the relative position of the searched string (like that it's at the beginning or end of a string), an anchor can avoid much backtracking and searching. | [reply] [d/l] [select] |
|
I'd expect that the unpack is the faster method to split the fields, but after inserting the following code in the above benchmark, in the average "dirunpk" (direct unpack) took the same amount of time than the "repeat" test.
#!perl
use v5.10;
use strict;
use warnings;
use Benchmark qw(:all);
my $results = timethese( 1e6,
{
repeat => sub{
my $t1 = '20090123';
$t1 =~ /(\d\d\d\d)(\d\d)(\d\d)/;
my ($y1,$m1,$d1) = ($1,$2,$3);
},
range => sub{
my $t2 = '20090123';
$t2 =~ /(\d{4})(\d{2})(\d{2})/;
my ($y2,$m2,$d2) = ($1,$2,$3);
},
chkunpk => sub{
my $t3 = '20090123';
$t3 =~ m/([0-9]{8})/;
my ($y3,$m3,$d3) = unpack "A4 A2 A2", $1;
},
dirunpk => sub{
my $t3 = '20090123';
my ($y4,$m4,$d4) = unpack "A4 A2 A2", $t3;
},
isook => sub{
my $t5 = '20090123';
$t5 =~ /(....)(..)(..)/;
my ($y5,$m5,$d5) = ($1,$2,$3);
},
} );
cmpthese( $results ) ;
__END__
1st run:
Benchmark: timing 1000000 iterations of chkunpk, dirunpk, isook, range
+, repeat...
chkunpk: 4 wallclock secs ( 3.11 usr + 0.00 sys = 3.11 CPU) @ 32
+1646.83/s (n=1000000)
dirunpk: 2 wallclock secs ( 2.06 usr + 0.00 sys = 2.06 CPU) @ 48
+4966.05/s (n=1000000)
isook: 1 wallclock secs ( 1.95 usr + 0.00 sys = 1.95 CPU) @ 51
+2032.77/s (n=1000000)
range: 3 wallclock secs ( 2.16 usr + 0.00 sys = 2.16 CPU) @ 46
+3821.89/s (n=1000000)
repeat: 2 wallclock secs ( 1.97 usr + 0.00 sys = 1.97 CPU) @ 50
+8130.08/s (n=1000000)
Rate chkunpk range dirunpk repeat isook
chkunpk 321647/s -- -31% -34% -37% -37%
range 463822/s 44% -- -4% -9% -9%
dirunpk 484966/s 51% 5% -- -5% -5%
repeat 508130/s 58% 10% 5% -- -1%
isook 512033/s 59% 10% 6% 1% --
2nd run:
Benchmark: timing 1000000 iterations of chkunpk, dirunpk, isook, range
+, repeat...
chkunpk: 2 wallclock secs ( 3.11 usr + 0.00 sys = 3.11 CPU) @ 32
+1646.83/s (n=1000000)
dirunpk: 3 wallclock secs ( 2.05 usr + 0.00 sys = 2.05 CPU) @ 48
+8519.79/s (n=1000000)
isook: 2 wallclock secs ( 1.98 usr + 0.00 sys = 1.98 CPU) @ 50
+4032.26/s (n=1000000)
range: 1 wallclock secs ( 2.11 usr + 0.00 sys = 2.11 CPU) @ 47
+4158.37/s (n=1000000)
repeat: 3 wallclock secs ( 2.06 usr + 0.00 sys = 2.06 CPU) @ 48
+4966.05/s (n=1000000)
Rate chkunpk range repeat dirunpk isook
chkunpk 321647/s -- -32% -34% -34% -36%
range 474158/s 47% -- -2% -3% -6%
repeat 484966/s 51% 2% -- -1% -4%
dirunpk 488520/s 52% 3% 1% -- -3%
isook 504032/s 57% 6% 4% 3% --
The faster way seems to be using the capture made of dots as in "isook", as the OP said that the string IS a date in ISO format.
UPDATE: Name of tests changed (for readability) and comparison table added. Results are for two consecutive runs.
| [reply] [d/l] [select] |
|
I'm not sure why you have the regex engine check if each character is not a newline in isook. Use the "s" modifier!
Interesting about pack. I heard the overhead to start the regex engine went up in 5.10, but it seems rather minor when put into perspective. ...except I can't replicate your results.
use strict;
use warnings;
use Benchmark qw(:all);
print("This is Perl $]\n");
my %tests = (
repeat => 'my ($y,$m,$d) = $date =~ /(\\d\\d\\d\\d)(\\d\\d)(\\d\\d
+)/;',
range => 'my ($y,$m,$d) = $date =~ /(\\d{4})(\\d{2})(\\d{2})/;',
isook => 'my ($y,$m,$d) = $date =~ /(....)(..)(..)/s;',
unpack => 'my ($y,$m,$d) = unpack "A4 A2 A2", $date;',
);
# These don't result in any opcodes.
$_ = 'use strict; use warnings; our $date; '.$_
for values(%tests);
our $date = '20091202';
my $results = cmpthese(-3, \%tests);
This is Perl 5.010000
Rate range repeat isook unpack
range 405773/s -- -6% -8% -46%
repeat 432956/s 7% -- -2% -43%
isook 441233/s 9% 2% -- -42%
unpack 757010/s 87% 75% 72% --
This is Perl 5.010000
Rate range isook repeat unpack
range 398141/s -- -7% -7% -47%
isook 427913/s 7% -- -0% -43%
repeat 429311/s 8% 0% -- -43%
unpack 751802/s 89% 76% 75% --
This is Perl 5.010000
Rate range repeat isook unpack
range 415595/s -- -7% -8% -45%
repeat 445365/s 7% -- -1% -41%
isook 449974/s 8% 1% -- -40%
unpack 754290/s 81% 69% 68% --
The faster way seems to be using the capture made of dots as in "isook"
You haven't shown that. Any difference less than 5% should be ignored. It's within the error margin.
| [reply] [d/l] [select] |
|
|
|
|
|
|
substr => sub{
my $t2 = '20090123';
my $year =substr ($t2,0,4);
my $mon = substr ($t2,4,2);
my $day = substr ($t2,6,2);
die if (length($t2)!= 8);
die if ($t2 =~ /\D/);
},
#!perl
use v5.10;
use strict;
use warnings;
use Benchmark qw(:all);
my $results = timethese( 1e6,
{
repeat => sub{
my $t1 = '20090123';
$t1 =~ /(\d\d\d\d)(\d\d)(\d\d)/;
my ($y1,$m1,$d1) = ($1,$2,$3);
},
range => sub{
my $t2 = '20090123';
$t2 =~ /(\d{4})(\d{2})(\d{2})/;
my ($y2,$m2,$d2) = ($1,$2,$3);
},
substr => sub{
my $t2 = '20090123';
my $year =substr ($t2,0,4);
my $mon = substr ($t2,4,2);
my $day = substr ($t2,6,2);
},
chkunpk => sub{
my $t3 = '20090123';
$t3 =~ m/([0-9]{8})/;
my ($y3,$m3,$d3) = unpack "A4 A2 A2", $1;
},
dirunpk => sub{
my $t3 = '20090123';
my ($y4,$m4,$d4) = unpack "A4 A2 A2", $t3;
},
isook => sub{
my $t5 = '20090123';
$t5 =~ /(....)(..)(..)/;
my ($y5,$m5,$d5) = ($1,$2,$3);
},
} );
cmpthese( $results ) ;
__END__
output:
Benchmark: timing 1000000 iterations of chkunpk, dirunpk, isook, range
+, repeat, substr...
chkunpk: 4 wallclock secs ( 5.05 usr + 0.00 sys = 5.05 CPU) @
198137.51/s (n=1000000)
dirunpk: 2 wallclock secs ( 3.31 usr + 0.00 sys = 3.31 CPU)
@ 301841.23/s (n=1000000)
isook: 3 wallclock secs ( 3.08 usr + 0.00 sys = 3.08 CPU) @
324886.29/s (n=1000000)
range: 4 wallclock secs ( 3.23 usr + 0.00 sys = 3.23 CPU) @
309214.59/s (n=1000000)
repeat: 4 wallclock secs ( 3.03 usr + 0.00 sys = 3.03 CPU) @
329924.12/s (n=1000000)
substr: 2 wallclock secs ( 1.22 usr + 0.00 sys = 1.22 CPU) @
820344.54/s (n=1000000)
Rate chkunpk dirunpk range isook repeat substr
chkunpk 198138/s -- -34% -36% -39% -40% -76%
dirunpk 301841/s 52% -- -2% -7% -9% -63%
range 309215/s 56% 2% -- -5% -6% -62%
isook 324886/s 64% 8% 5% -- -2% -60%
repeat 329924/s 67% 9% 7% 2% -- -60%
substr 820345/s 314% 172% 165% 153% 149% --
| [reply] [d/l] [select] |
|
| [reply] [d/l] |
|
| [reply] |
|
| [reply] |
|
|
|
|
Re: better (faster) way of writing regexp
by BioLion (Curate) on Dec 02, 2009 at 14:17 UTC
|
Not quite what you asked for - but you could take a look at the range of Date modules on cpan for parsing date strings, they might be faster, and will almost definitely lead to more readable code!
Just a something something...
| [reply] |
|
|