A Regex to identify Regex's / Compiling a regex

[knowmad]

Hi monks,

I am working on some modifications to the Data::FormValidator module and have a couple of questions about regex's. The first is regarding a gnarly regex which I am trying to understand and the second is about compiling a regex into an anonymous subroutine.

I've put the regex which matches regex's into extended format and started adding some comments to figure out what this beast is doing. Any additional info would be helpful.

$constraint =~ m@
    ^\s*        # skip all whitespace at beginning
    (
        /.+/            # capture the custom regex
        |m(.).+\2    # does this capture the ops? What does the |m do?
+ why is the (.) and \2 in there?
    )
    [cgimosx]*    # ?? don't understand what this is for. what are we 
+trying to match? I thought the ops were matched above
    \s*$        # skip all whitespace at end
    @x

The Data::FormValidator allows users to submit custom regex's for checking input. It does this by compiling the regex into an anonymous subroutine (see code below). I have discovered a curious anomaly in the evaluation phase of the subroutine when dealing with a regex that contains a \\ (an escaped backslash). Basically, one regex ($constraint_good) will compile and run while the second will not compile. The only difference between the two is the placement of the \\. The working regex puts this at the beginning; the bad one has this at the end (don't even ask how I figured this out; one of those flukes). See the code below for details.

my $input = 'http://www.knowmad.com/';
my $constraint_good = '/[\\!@#%&_:\$\^\*\(\)\+\.\/]+/';
my $constraint_bad = '/[!@#%&_:\$\^\*\(\)\+\.\/]+\\/';


if ( $constraint_good =~ m@^\s*(/.+/|m(.).+\2)[cgimosx]*\s*$@ ) {
    my $sub = eval 'sub { $_[0] =~ '. $constraint_good .  '}';
        die "Error compiling regular expression $constraint_good: $@" 
+if
            $@;
        print "matched\n";
} else {
    print "no good.\n"
}

if ( $constraint_bad =~ m@^\s*(/.+/|m(.).+\2)[cgimosx]*\s*$@ ) {
    my $sub = eval 'sub { $_[0] =~ '. $constraint_bad .  '}';
        die "Error compiling regular expression $constraint_bad:
            $@" if $@;
        print "matched\n";
} else {
    print "no good.\n"
}

When you run this snippet you should receive the following (tested under Perl 5.6.1 on WinNT4 and Debian 3.0):

matched
Error compiling regular expression /[!@#%&_:\$\^\*\(\)\+\.\/]+\/: Sear
+ch pattern not terminated at (eval 2) line 1.

Your wisdom or comments are greatly appreciated.

William

[BrowserUk]

$constraint =~ m@
  ^\s*                              # skip all whitespace at beginning
  (                                 # start capturing
    /.+/                            # match a '/' any number of anycha
+r '/'
    |                               # or
    m                               # char 'm' (start of a match)
    (.)                             # capture the delimiter
    .+                              # skip all the stuff 
    \2                              # upto the second occurance of the
+ delimiter
  )                                 #end capture
  [cgimosx]*                        # match zero or more match options
  \s*$                              # skip white space
  @x

Basically, the regex validates regex of either /????/ or m/?????/cgimosx form.

I think your problem is that when the \ is at the end, it is escaping the the final forward slash and that won't match the regex validating regex. Try using the m!! form with a different delimiter. Should help.

---

Cor! Like yer ring! ... HALO dammit! ... 'Ave it yer way! Hal-lo, Mister la-de-da. ... Like yer ring!

[knowmad]

Ahhh, now I see! Thanks for editing that regex for me. I like your explanation of why using a \\ at the end fails. However, if the regex did not match, then a compile error would not occur as it would not enter the if statement.

Following your suggestion of using the m!! format works. So the problem lies in how the regex is getting compiled. I did a bit more testing and discovered that using the \\ fails when it is next to a \/ combination (on either side). It fails if I use the // or m// style. At this point it's mostly a matter of curiousity but an interesting anomaly within either the eval function or the Perl regex engine.

Thanks, William

[bart]

At this point it's mostly a matter of curiousity but an interesting anomaly within either the eval function or the Perl regex engine.

It's not. It's just that you're trying to match a backslash, for which you need a double backslash in the regex. But to achieve that, you need to type 4 backslashes in a singlequotish string. It's what's in the string that matters for the regex engine, not what's in your source code.

A backslash only disappears in such a string, if it's in front of a backlslash or a string delimiter (here "'").

[BrowserUk]

I don't think there is any mystery here. Although the regexes are originally within single quotes and therefor not interpolated, in the eval they are being concatenated with the rest of the sub statement. The eval see's it just as a string. When the interpreter tries to evaluate the combined string the final delimiter of the match is escaped, hence the error, Search pattern not terminated.

It get's into the if statement ok, but its when its concatenated prior to evaluation where the problem arises.

That's why using m!! cures the problem.

I'll add... I think, cos I'm not great with regexes.

---

Cor! Like yer ring! ... HALO dammit! ... 'Ave it yer way! Hal-lo, Mister la-de-da. ... Like yer ring!

[zigdon]

$constraint =~ m@
   ^\s*        # skip all whitespace at beginning
   (
       /.+/            # capture the custom regex
       |m(.).+\2    # does this capture the ops? What does the |m do? 
+why is the (.) and \2 in there?
   )
   [cgimosx]*    # ?? don't understand what this is for. what are we t
+rying to match? I thought the ops were matched above
   \s*$        # skip all whitespace at end
   @x

regexes can be in a number of ways. one, is the familiar /regex/flags. Another, is with the m// operator, which can have any thing instead of the "/". I think there are some other ways as well, but those are not matched by this regex.

the m(.).+\2 part of the regex is for matching m// type regexes. it says "Match an 'm', followed by any character (the delimiter), followed by at least one character (the regex itself), until you reach the delimiter again". a "\2" is a of using backreferences in regexes. It matches "whatever was in the 2nd set of ()s".

See man perlop for more details.

-- Dan

[sauoq]

You are using \2 to match the second delimiter but the second delimiter is not required to be the same character as the first one. Consider m{braces as delimiters} for example. Other pairs to watch for include m[ ], m( ), and even m< >.

-sauoq
"My two cents aren't worth a dime.";

[mirod]

Here is what I use sometimes, which is not perfect as you can't use brackets inside a regexp, I have to add level counting (comments and patches welcome!). You could also have a look at Regexp::Common::Balanced.

#!/bin/perl -w
use strict;

my %brace=( '[' => ']', '(' =>')', '{' => '}', '<' => '>');
my $not_brace_class= '[^\\' . join( '\\', keys %brace) . ']';

my $SIMPLE_REGEXP = "(($not_brace_class)(?:(\\\\.|.)*?)\\2)";

my @regexp= ($SIMPLE_REGEXP);
while( my( $open, $close)= each( %brace))
  { push @regexp, "(\\$open(?:\\\\.|.)*?\\$close)"; }


while( <DATA>)
  { chomp;
    my( $exp, $expected_match)= split /\t+/;
    print "$exp: ";
    foreach my $regexp (@regexp)
      { if( $exp=~ m{^$regexp}) 
          { my $found= $1; 
            if( $found eq $expected_match) 
          { print "REGEXP OK "; }
        else
          { print "REGEXP matches $found instead of $expected_match (r
+egexp is $regexp)"; }
        last;
      }
      }
    print "\n";
  }

__DATA__
/toto/        /toto/
#toto#        #toto#
{toto}        {toto}
/to\to/        /to\to/
/to\/to/    /to\/to/
#to\to#        #to\to#
#to\#to#    #to\#to#
[to\to]        [to\to]
[to\]to]    [to\]to]
[to[to]]    [to[to]]
/toto/no    /toto/
#toto#no    #toto#
{toto}no    {toto}
/to\to/no    /to\to/
/to\/to/no    /to\/to/
#to\to#no    #to\to#
#to\#to#no    #to\#to#
[to\to]no    [to\to]
[to\]to]no    [to\]to]
[to[to]]no    [to[to]]
/toto/no/    /toto/
#toto#no#    #toto#
{toto}no{no    {toto}
{toto}no{}no    {toto}
{toto}no}no    {toto}
{toto}no{    {toto}
{toto}no{}    {toto}
{toto}no}    {toto}
/to\to/no/    /to\to/
/to\to/no\/    /to\to/
/to\/to/no/    /to\/to/
#to\to#no\#    #to\to#
#to\#to#no#    #to\#to#
[to\to]no[    [to\to]
[to\]to]no]    [to\]to]
[to[to]]no[]    [to[to]]