Re: Hypothesis: some magic string optimalization in perl kills my server from time to time

[tybalt89]

I had what I think is a similar problem. When I changed my whitespace eater to

my $ws = qr/(?:#.*|\s+)*+/; # white space

the problem went away. Try it and let us know. Also something like

my $ws = qr/(?:#.*+|\s++)*/; # white space

should work, or

my $ws = qr/(?>(?:#.*|\s+)*)/; # white space

anything that will not give back after matching.

[leszekdubiel]

I have changed to possesive version -- unfortunatelly it didn't help... Still the same effect

$$a =~ /\G(?:\s++|#.*+)*+\(/gc or croak "parenthesis expected"

[tybalt89]

And do you have a small input case where the slowness happens (so I can do testing at home :) ?

[leszekdubiel]

Here is the file -- 200 kilobytes (... small one):

download from Poland

download from we transfer

There is minimal test and description. Below is how I run tests. See -- one is half a seconds, and other is 20 seconds.



# ./show_me_the_problem.sh 


****************************
here are the two programs:
-rwxr-xr-x 1 root root 177 wrz 30 17:57 run_fast.pl
-rwxr-xr-x 1 root root 190 wrz 30 17:57 run_slow.pl


****************************
the difference is: 
10c10,11
< my $lst = list_extr(\( "(" . $inp . ")" )); 
---
> my $par = "(" . $inp . ")"; 
> my $lst = list_extr(\$par); 


****************************
data is the same... let's run them...
time ./run_fast.pl | wc -l
9506

real    0m0.476s
user    0m0.456s
sys    0m0.016s


****************************
time ./run_slow.pl | wc -l
9506

real    0m22.591s
user    0m22.556s
sys    0m0.012s


****************************
seee... programs are almost the same... but their time is totally dife
+erent

[tybalt89]

Which regex does it stop at?

[leszekdubiel]

The program doesn't stop at any of regexpes. It finishes computations sucessfully, but sometimes it takes quite short, and sometimes it takes very long.

For example on 2Mb of data, when I call "list_extr" like this:

list_extr(\( "(" . $big_string . ")" ));

it takes only half a second. But if I call like this:

my $ttt = "(" . $big_string . ")";
list_extr(\$ttt);

it takes 20 seconds (slower 40x times). Why? Both ways look the same -- make string with parentheses around old string, take reference to it, do same computations. So if they are the same why the difference in time? I think perl is doing some magic here, but only someone more expierenced would tell.