Skipping special tags in regexes

[fletcher_the_dog]

Hi All, I have files that have some special markers in them something like this:
<5b>I <5c>like <5d>tacos
I was wondering if there was anyway possible to make the regex engine ignore these tags without deleting them? For example, if I had a regex like this:

s/(\w+)\s+tacos/$1 yummy tacos/g;

I would want it to change the above phrase to:
<5b>I <5c>like yummy <5d>tacos
I realize I could change the regex to:

s/(\w+)\s+(<\w\w>)?tacos/$1 yummy $2tacos/g;

but my script requires quite a few complicated regexes substitutions and having optional "(<\w\w>)?"s everywhere will make the regexes very ugly and make it hard to keep track of which capturing parenthesis are catching what.
I was wondering if there was a way to make the regex engine see the a string something like this where it would do its matching just against the first element in each sub array:

$string = [
  ['I','<5b>'],
  [' '],
  ['l','<5c>'],
  ['i'],
  ['k'],
  ['e'],
  [' '],
  ['t','<5d>'],
  ['a'],
  ['c'],
  ['o'],
  ['s']
]

Another possibilty I have considered and tried was erasing all the tags and then after I was done with all the substitutions doing a diff and trying to reconstruct where the tags should go. This actually works but was way too slow.

[dragonchild]

Build dynamic regexes. Maybe something like:

my $regex = 's/(\w+)\s+tacos/$1 yummy tacos/g';
$regex =~ s/(\\s\+)(\w)/$1(<\\w\\w>)$2/g;       # Handle first part of
+ substitution
my $match = $2;
$regex =~ s/ $match/\$2$match/g;                # Handle second part o
+f substitution
eval "$regex";

That would handle the transform from the first to the second. Ideally, you would re-evaluate your regexes and build them using some regex builder. The builder would handle the optional tags and making sure they stayed in after the substitutions.

[thospel]

The wanted solution is not well specified though. In your example, it would seem equally valid to return:
<5b>I<5c>like <5d>yummy tacos
(corresponding to a different place to put the (<\w\w>)?).You'll have to state this doesn't matter or state a way to resolve this ambiguity.

[fletcher_the_dog]

That is good point. The tags are associated with the word immediately following them, so it would be nice if I could associate them with the first character of that word.

[Abigail-II]

#!/usr/bin/perl

use strict;  
use warnings;

my $tag  = '(?:<[^>]*>)';
sub word {
    my $word = shift;
    qr /$tag*$word/;
}

while (<DATA>) {
    s/((??{ word 'tacos' }))/yummy $1/g;
    s/((??{ word 'salad' }))/green $1/g;
    print;
}

__DATA__
I like tacos.
<5b>I <5c>like <5d>tacos.
I like a salad.
<foo>I <foo>like <foo>a <bar><baz><foo>salad.
<foo>I <foo>like <foo>a <bar><baz><foo>salad <bup>with <bobob>tacos.


I like yummy tacos.
<5b>I <5c>like yummy <5d>tacos.
I like a green salad.
<foo>I <foo>like <foo>a green <bar><baz><foo>salad.
<foo>I <foo>like <foo>a green <bar><baz><foo>salad <bup>with yummy <bo
+bob>tacos.

Abigail

[flounder99]

Is there a reason you are using the /(??{ })/ code block other than just to make a more general solution? This seems to work just fine.

#!/usr/bin/perl

use strict;  
use warnings;

my $tag  = '(?:<[^>]*>)';

while (<DATA>) {
    s/($tag*tacos)/yummy $1/g;
    s/($tag*salad)/green $1/g;
    print;
}

__DATA__
I like tacos.
<5b>I <5c>like <5d>tacos.
I like a salad.
<foo>I <foo>like <foo>a <bar><baz><foo>salad.
<foo>I <foo>like <foo>a <bar><baz><foo>salad <bup>with <bobob>tacos.


I like yummy tacos.
<5b>I <5c>like yummy <5d>tacos.
I like a green salad.
<foo>I <foo>like <foo>a green <bar><baz><foo>salad.
<foo>I <foo>like <foo>a green <bar><baz><foo>salad <bup>with yummy <bo
+bob>tacos.

[Abigail-II]

Is there a reason you are using the /(??{ })/ code block other than just to make a more general solution?

Because I was expecting the OP to refine his question, and come up with a slightly different definition of a "word" (perhaps it needed trailing tags as well, or not more than 2 tags, whatever). Then I only need to change the sub, and not every regex using it.

I think my solution is more general than yours, but the effect is the same.

Abigail

[BrowserUk]

You might consider creating your own custom tag. This example is probably not well thought through, it's basically just a tweaking of the example given at the end of perlre, but I'd never tried this before so I did what came easy:)

#! perl -slw
use strict;
package CustomReTag;
use overload;

sub import {
shift;
    die "No argument to customre::import allowed" if @_;
    overload::constant 'qr' => \&convert;
}

sub invalid { die "/$_[0]/: invalid escape '\\$_[1]'"}

my %rules = (
    '\\' => '\\',
    'Y|' => qr/<\w\w>/
);

sub convert {
    my $re = shift;
    $re =~ s{ \\ ( \\ | Y . ) }
            { $rules{$1} or invalid($re,$1) }sgex;
    return $re;
}

package main;

my $s = '<5b>I <5c>like <5d>tacos';

my $re = CustomReTag::convert '(\Y|tacos)';

$s =~ s[$re][yummy $1]g;

print $s;

__END__
P:\test>junk
<5b>I <5c>like yummy <5d>tacos

[delirium]

Another possibility is to not rely on regexes. I'm not sure what your specs for searching and replacing are, but if they are narrowed down to prefixing words with other words, as in your example, you could get away with building a custom function and passing it the line to change, the word to look for, and what to prefix it with, e.g.:

#!/usr/bin/perl -w
use strict;

sub prefix    {
    my ($line, $look_for, $prefix, $flip) = @_;
    return $line unless $look_for && $line =~ /$look_for/;
    my @arr = split /(<[^>]*>)/, $line;
    for my $cnt (0..$#arr)    {
        if ($arr[$cnt] =~ /$look_for/)    {
            $arr[$cnt-2] .= $prefix if $cnt > 1 && !$flip;
            $arr[$cnt+2] = $prefix . ' ' . $arr[$cnt+2] if $cnt < $#ar
+r-1 && $flip;
        }
    }
    return join '', @arr;
}


my $line = "<5b>I <5c>like <5d>tacos\n";
print prefix ($line, 'tacos', 'yummy');
print prefix ($line, 'like', 'yummy', 1);

__OUTPUT__
<5b>I <5c>like yummy<5d>tacos
<5b>I <5c>like <5d>yummy tacos

[fletcher_the_dog]

unfortunately, I have many regexes that are not fixed strings and require all the powers of regexes

[BUU]

My best guess would be to actually attempt to parse it some how, and store the total thing in some sort of datastructure. The simples would be a hash of the form tag => string, then you could just iterate over the values of the hash to ignore the tags, and vice versa. How you would actually parse this string is a bit beyond me, if the tags are truly as simple as you depict here then it should be fairly simple to just use a regex /<5\w>\w+/ or something, but beyond that you would have to look at some of the parsers on cpan.

[CountZero]

If the tags are associated with the word directly following them (without any intervening whitespace), then you could split the sentence on whitespace and then split of each tag from the word following it.

It would then be trivial to build a data-structure you could use as a basis to put the tags back in after the regex has done its thing with the "untagged" sentence.

[fletcher_the_dog]

The only problem is that each word in a string may not be unique, so you couldn't just plop things in a hash. Also the regexes might introduce new words someplace in the string that alreay existed in the string somewhere else. That why I have tried using diffing, but it was just too slow.