Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?

Re: Regex confusion

by MarkusLaker (Beadle)
on Dec 24, 2004 at 21:54 UTC ( [id://417361]=note: print w/replies, xml ) Need Help??

in reply to Regex confusion

I wish I could remember who originally said this: regular expressions may be greedy, but they are not into deferred gratification.

Let's consider what happens when we apply this regex --

^int\s+(\w+)\s* (?:^\s*Number\sof\sFlaps:\s(\d+)\s*)? (?:^\s*IP\sAddress:?\s(\S*)\s*)?

-- to this string:

int A Number of Flaps: 6 IP Address:

The \s* in the first line of the regex eats up any spaces at the end of the first line of input text, plus the newline, plus the indent at the start of the second line. Since the rest of the regex is inside (?: ... )? brackets, the regex can match successfully without backtracking, and that's exactly what it does -- without matching the rest of the input string.

Now let's remove the question marks at the end of the second and third lines of the regex. This forces the regex engine to match the text inside the brackets if the match is to succeed. This in turn forces the regex engine to backtrack when it tries to match that first \s*, so that the \s* matches any trailing space at the end of the first line of text, and one newline, and nothing more. This leaves the ^ and the rest of the second and third lines of the regex to match successfully.

To solve your problem -- to make the last two lines of data optional -- you need to add newlines to the regex, like this:

^int\s+(\w+)\s*\n (?:^\s*Number\sof\sFlaps:\s(\d+)\s*\n)? (?:^\s*IP\sAddress:?\s(\S*)\s*\n)?

In each case, the newline stops the preceding \s* from slurping up the indenting whitespace at the start of the next line. This makes it possible for the text inside the brackets to match without backtracking.

Like most things related to regexes, this is hard to explain. The principle I want to get across is that every term followed by a star will match greedily (i.e. as many times as possible); and, if Perl can make the regex match without backtracking, it will. This is what was happening with your original version of the regex. The regex engine is prepared to backtrack in order to make the whole regex match, but not in order to match optional items. What the added newlines do in our example is to make sure that the regex engine has reached a point in the input text where the optional brackets can match without the need for backtracking.

I don't feel I've explained this very well, so feel free to come back with questions about it -- but I'll be off the Net for a few days, so I can't reply promptly.

Merry Christmas to all.


Replies are listed 'Best First'.
Re^2: Regex confusion
by scottb (Scribe) on Dec 25, 2004 at 06:34 UTC
    Thank you very much! It makes perfect sense. Actually to be honest I feel silly now that you've explained it to me as you have because it's the kind of thing I would have expected myself to catch.

    Have a great Christmas, and thanks again.
    - Scott

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://417361]
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others imbibing at the Monastery: (4)
As of 2024-04-22 11:58 GMT
Find Nodes?
    Voting Booth?

    No recent polls found