Re^2: better (faster) way of writing regexp

in reply to Re: better (faster) way of writing regexp
in thread better (faster) way of writing regexp

I'd expect that the unpack is the faster method to split the fields, but after inserting the following code in the above benchmark, in the average "dirunpk" (direct unpack) took the same amount of time than the "repeat" test.

#!perl
use v5.10;
use strict;
use warnings;
use Benchmark qw(:all);

my $results = timethese( 1e6,
  {
    repeat => sub{
      my $t1 = '20090123';
      $t1 =~ /(\d\d\d\d)(\d\d)(\d\d)/;
      my ($y1,$m1,$d1) = ($1,$2,$3);
    },
    range => sub{
      my $t2 = '20090123';
      $t2 =~ /(\d{4})(\d{2})(\d{2})/;
      my ($y2,$m2,$d2) = ($1,$2,$3);
    },
    chkunpk => sub{
      my $t3 = '20090123';
      $t3 =~ m/([0-9]{8})/;
      my ($y3,$m3,$d3) = unpack "A4 A2 A2", $1;
    },
    dirunpk => sub{
      my $t3 = '20090123';
      my ($y4,$m4,$d4) = unpack "A4 A2 A2", $t3;
    },
    isook => sub{
      my $t5 = '20090123';
      $t5 =~ /(....)(..)(..)/;
      my ($y5,$m5,$d5) = ($1,$2,$3);
    },
  } );

cmpthese( $results ) ;

__END__

1st run:

Benchmark: timing 1000000 iterations of chkunpk, dirunpk, isook, range
+, repeat...
   chkunpk:  4 wallclock secs ( 3.11 usr +  0.00 sys =  3.11 CPU) @ 32
+1646.83/s (n=1000000)
   dirunpk:  2 wallclock secs ( 2.06 usr +  0.00 sys =  2.06 CPU) @ 48
+4966.05/s (n=1000000)
     isook:  1 wallclock secs ( 1.95 usr +  0.00 sys =  1.95 CPU) @ 51
+2032.77/s (n=1000000)
     range:  3 wallclock secs ( 2.16 usr +  0.00 sys =  2.16 CPU) @ 46
+3821.89/s (n=1000000)
    repeat:  2 wallclock secs ( 1.97 usr +  0.00 sys =  1.97 CPU) @ 50
+8130.08/s (n=1000000)
            Rate chkunpk   range dirunpk  repeat   isook
chkunpk 321647/s      --    -31%    -34%    -37%    -37%
range   463822/s     44%      --     -4%     -9%     -9%
dirunpk 484966/s     51%      5%      --     -5%     -5%
repeat  508130/s     58%     10%      5%      --     -1%
isook   512033/s     59%     10%      6%      1%      --

2nd run:

Benchmark: timing 1000000 iterations of chkunpk, dirunpk, isook, range
+, repeat...
   chkunpk:  2 wallclock secs ( 3.11 usr +  0.00 sys =  3.11 CPU) @ 32
+1646.83/s (n=1000000)
   dirunpk:  3 wallclock secs ( 2.05 usr +  0.00 sys =  2.05 CPU) @ 48
+8519.79/s (n=1000000)
     isook:  2 wallclock secs ( 1.98 usr +  0.00 sys =  1.98 CPU) @ 50
+4032.26/s (n=1000000)
     range:  1 wallclock secs ( 2.11 usr +  0.00 sys =  2.11 CPU) @ 47
+4158.37/s (n=1000000)
    repeat:  3 wallclock secs ( 2.06 usr +  0.00 sys =  2.06 CPU) @ 48
+4966.05/s (n=1000000)
            Rate chkunpk   range  repeat dirunpk   isook
chkunpk 321647/s      --    -32%    -34%    -34%    -36%
range   474158/s     47%      --     -2%     -3%     -6%
repeat  484966/s     51%      2%      --     -1%     -4%
dirunpk 488520/s     52%      3%      1%      --     -3%
isook   504032/s     57%      6%      4%      3%      --
[download]

The faster way seems to be using the capture made of dots as in "isook", as the OP said that the string IS a date in ISO format.

UPDATE: Name of tests changed (for readability) and comparison table added. Results are for two consecutive runs.

Comment on Re^2: better (faster) way of writing regexp Select or Download Code

Replies are listed 'Best First'.
Re^3: better (faster) way of writing regexp by ikegami (Patriarch) on Dec 02, 2009 at 18:07 UTC
I'm not sure why you have the regex engine check if each character is not a newline in `isook`. Use the "s" modifier! Interesting about pack. I heard the overhead to start the regex engine went up in 5.10, but it seems rather minor when put into perspective. ...except I can't replicate your results. use strict; use warnings; use Benchmark qw(:all); print("This is Perl $]\n"); my %tests = ( repeat => 'my ($y,$m,$d) = $date =~ /(\\d\\d\\d\\d)(\\d\\d)(\\d\\d +)/;', range => 'my ($y,$m,$d) = $date =~ /(\\d{4})(\\d{2})(\\d{2})/;', isook => 'my ($y,$m,$d) = $date =~ /(....)(..)(..)/s;', unpack => 'my ($y,$m,$d) = unpack "A4 A2 A2", $date;', ); # These don't result in any opcodes. $_ = 'use strict; use warnings; our $date; '.$_ for values(%tests); our $date = '20091202'; my $results = cmpthese(-3, \%tests); [download] This is Perl 5.010000 Rate range repeat isook unpack range 405773/s -- -6% -8% -46% repeat 432956/s 7% -- -2% -43% isook 441233/s 9% 2% -- -42% unpack 757010/s 87% 75% 72% -- This is Perl 5.010000 Rate range isook repeat unpack range 398141/s -- -7% -7% -47% isook 427913/s 7% -- -0% -43% repeat 429311/s 8% 0% -- -43% unpack 751802/s 89% 76% 75% -- This is Perl 5.010000 Rate range repeat isook unpack range 415595/s -- -7% -8% -45% repeat 445365/s 7% -- -1% -41% isook 449974/s 8% 1% -- -40% unpack 754290/s 81% 69% 68% -- [download] The faster way seems to be using the capture made of dots as in "isook" You haven't shown that. Any difference less than 5% should be ignored. It's within the error margin.	[reply] [d/l] [select]
Re^4: better (faster) way of writing regexp by JavaFan (Canon) on Dec 02, 2009 at 18:19 UTC
`# These don't result in any opcodes. $_ = "use strict; use warnings; our $date; $_" for values(%tests);` [download] Indeed no opcodes, as it doesn't compile. You're interpolating a variable that hasn't been introduced to perl yet.	[reply] [d/l]
Re^5: better (faster) way of writing regexp by ikegami (Patriarch) on Dec 02, 2009 at 18:33 UTC
~~It doesn't later either. In other words, they're not executed during Benchmarking.~~ Ah, I misread earlier. I had posted the wrong copy of the code. Fixed.	[reply]
Re^4: better (faster) way of writing regexp by Marshall (Canon) on Dec 03, 2009 at 07:52 UTC
Ikegami, I am curious if you can replicate my results with the substr idea plugged into your benchmark code? Update: the reason I ask is that I know you have a very fast 64 bit machine and there could be some differences between my much slower, older 32 bit machine. use strict; use warnings; use Benchmark qw(:all); print("This is Perl $]\n"); my %tests = ( repeat => 'my ($y,$m,$d) = $date =~ /(\\d\\d\\d\\d)(\\d\\d)(\\d\\d +)/;', range => 'my ($y,$m,$d) = $date =~ /(\\d{4})(\\d{2})(\\d{2})/;', isook => 'my ($y,$m,$d) = $date =~ /(....)(..)(..)/s;', unpack => 'my ($y,$m,$d) = unpack "A4 A2 A2", $date;', substr => 'my $y = substr($date,0,4);my $m = substr($date,4,2);my +$d = substr($date,6,2);' ); # These don't result in any opcodes. $_ = 'use strict; use warnings; our $date; '.$_ for values(%tests); our $date = '20091202'; my $results = cmpthese(-3, \%tests); __END__ This is Perl 5.010000 Rate range isook repeat unpack substr range 151695/s -- -7% -8% -54% -85% isook 162964/s 7% -- -1% -50% -84% repeat 165314/s 9% 1% -- -49% -84% unpack 326977/s 116% 101% 98% -- -68% substr 1010101/s 566% 520% 511% 209% -- [download]	[reply] [d/l]
Re^5: better (faster) way of writing regexp by ikegami (Patriarch) on Dec 03, 2009 at 17:42 UTC
I know you have a very fast 64 bit machine Dream on! My 32-bit machine doesn't even have a virtual second core from hyperthreading. I'm not sure what my work machine is, but it's also 32-bit. My earlier run was on my work machine. This is on my work machine too: `This is Perl 5.010000 Rate range repeat isook unpack substr range 416206/s -- -6% -8% -45% -67% repeat 443742/s 7% -- -1% -41% -65% isook 450082/s 8% 1% -- -41% -64% unpack 756739/s 82% 71% 68% -- -40% substr 1257912/s 202% 183% 179% 66% -- This is Perl 5.010000 Rate range repeat isook unpack substr range 415726/s -- -5% -8% -47% -67% repeat 436462/s 5% -- -4% -44% -65% isook 454041/s 9% 4% -- -42% -64% unpack 779486/s 88% 79% 72% -- -38% substr 1262559/s 204% 189% 178% 62% --` [download] (Threaded 32-bit build on linux) Interesting.	[reply] [d/l]
Re^6: better (faster) way of writing regexp by Marshall (Canon) on Dec 03, 2009 at 22:28 UTC
Re^7: better (faster) way of writing regexp by ikegami (Patriarch) on Dec 03, 2009 at 22:38 UTC
Re^3: better (faster) way of writing regexp by Marshall (Canon) on Dec 03, 2009 at 07:11 UTC
If we are assuming that isook() is fine functionally, there is a much faster way. Don't use regex! use substr! I ran same benchmarks along with a new one, substr(). Code is below. My machine, (older Win XP running Perl 5.10) is considerably slower than the other machines posting results, but I would expect similar results in the ranking if you run the code below. Having said that, all these solutions run so fast that it is hard to see how this is going to make much difference, but that depends upon the application! This substr idea is also faster even with some simple format error tests like this snippet... This $1,$2,$3 stuff is "expensive". `substr => sub{ my $t2 = '20090123'; my $year =substr ($t2,0,4); my $mon = substr ($t2,4,2); my $day = substr ($t2,6,2); die if (length($t2)!= 8); die if ($t2 =~ /\D/); },` [download] #!perl use v5.10; use strict; use warnings; use Benchmark qw(:all); my $results = timethese( 1e6, { repeat => sub{ my $t1 = '20090123'; $t1 =~ /(\d\d\d\d)(\d\d)(\d\d)/; my ($y1,$m1,$d1) = ($1,$2,$3); }, range => sub{ my $t2 = '20090123'; $t2 =~ /(\d{4})(\d{2})(\d{2})/; my ($y2,$m2,$d2) = ($1,$2,$3); }, substr => sub{ my $t2 = '20090123'; my $year =substr ($t2,0,4); my $mon = substr ($t2,4,2); my $day = substr ($t2,6,2); }, chkunpk => sub{ my $t3 = '20090123'; $t3 =~ m/([0-9]{8})/; my ($y3,$m3,$d3) = unpack "A4 A2 A2", $1; }, dirunpk => sub{ my $t3 = '20090123'; my ($y4,$m4,$d4) = unpack "A4 A2 A2", $t3; }, isook => sub{ my $t5 = '20090123'; $t5 =~ /(....)(..)(..)/; my ($y5,$m5,$d5) = ($1,$2,$3); }, } ); cmpthese( $results ) ; __END__ output: Benchmark: timing 1000000 iterations of chkunpk, dirunpk, isook, range +, repeat, substr... chkunpk: 4 wallclock secs ( 5.05 usr + 0.00 sys = 5.05 CPU) @ 198137.51/s (n=1000000) dirunpk: 2 wallclock secs ( 3.31 usr + 0.00 sys = 3.31 CPU) @ 301841.23/s (n=1000000) isook: 3 wallclock secs ( 3.08 usr + 0.00 sys = 3.08 CPU) @ 324886.29/s (n=1000000) range: 4 wallclock secs ( 3.23 usr + 0.00 sys = 3.23 CPU) @ 309214.59/s (n=1000000) repeat: 4 wallclock secs ( 3.03 usr + 0.00 sys = 3.03 CPU) @ 329924.12/s (n=1000000) substr: 2 wallclock secs ( 1.22 usr + 0.00 sys = 1.22 CPU) @ 820344.54/s (n=1000000) Rate chkunpk dirunpk range isook repeat substr chkunpk 198138/s -- -34% -36% -39% -40% -76% dirunpk 301841/s 52% -- -2% -7% -9% -63% range 309215/s 56% 2% -- -5% -6% -62% isook 324886/s 64% 8% 5% -- -2% -60% repeat 329924/s 67% 9% 7% 2% -- -60% substr 820345/s 314% 172% 165% 153% 149% -- [download]	[reply] [d/l] [select]

In Section Seekers of Perl Wisdom