Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer

japhy and mystery

by japhy (Canon)
on May 06, 2001 at 21:39 UTC ( #78369=perlmeditation: print w/replies, xml ) Need Help??

I heard from a little bird that people were conspiring against me on Saturday evening, to help write a Perl book. There were two rumors I heard: Mastering Regular Expressions, 2nd Ed.; and the Perl Puzzle Book.

I would love to write a book about Perl. Even moreso, a book about Perl-compatiable regexes (PCREs). And even moreso, a book about all the wonderful things you can do with regexes in Perl. I don't think I could fill J. Friedl's shoes to produce the second edition of the Hip Owls book; I'm not well versed in other implementations of regexes, and I must say that the book was my first introduction to NFA and DFA. Before I read that book, I just used regexes. I didn't "know" them.

So here is my verdict. I want to write a regex book. I want to write about sexeger, and explain. I want to update the MRE chapter on Perl's regexes, since a lot has changed. I want to make a source of Perl regex knowledge other than perlre and perlretut. I think something like "Learning Perl Regexes" (complete with exercises and all) would be a good book to have. I think "The Perl Puzzle Book" would be an excellent idea too, and I could contribute to that more than just regex gems.

I'm a college student. I have classes to deal with (or I will next semester). But I won't drop off the face of the Perl-planet while I'm at college. Perhaps I could write a book in that time, with the help of some other fanatics. I'm up for the challenge. Who's with me?

(Note: the title of this node is "japhy and Mr. E", or "MRE".)

japhy -- Perl and Regex Hacker

Replies are listed 'Best First'.
Learning Perl's Regular Expressions
by japhy (Canon) on May 07, 2001 at 01:37 UTC
    Here is my vision. Please entertain it, if you would. If you haven't been able to tell, I'm serious, and I'm a regex fanatic. Learning Perl's Regular Expressions
    • Familiarize the reader with the tasks that regular expressions are intended to complete.
      • What is a regular expression (regex)?
      • What can they do?
      • What are some practical uses for them?
      • What are some impractical uses for them?
      • How can they solve daily programming problems?
    • Abstract thinking about regexes.
      • How to explain a regex's purpose.
      • How to tackle a given parsing problem with a regex.
      • How to say what you mean, and mean what you say.
      • What else is out there, apart from regexes.
    • Simple patterns.
      • Plain text: /hello world/
      • Safe patterns: /hello \Q$place\E/
      • Character classes: /[Hh]ello [Ww]orld/
      • Case-insensitivity: /hello world/i
      • Alternation: /hello|goodbye/
      • Matching something other than $_: $foo =~ /hello/
    • More "patterny" patterns.
      • Dot: /h.ll/
      • Quantifiers: /hello(?: +world)?/
      • Macros: /hello\s+world/
      • Anchors: /^hello,?\s+world$/
      • Alternate m// characters: m!a/b/c!
    • Getting something back.
      • Capturing: /hello,?\s+(\w+)/
      • Back-references: /\b(\w)\w+(\1)\b/
      • Global matching: /(\d+)/g
      • Regex-y functions: split(), grep(), map()
      • Useful variables: $1, @-, @+
      • Not-so-useful variables: $`, $&, $'
    • More modifiers.
      • Modifying .: /a.b/s
      • Modifying ^ and $: /^foo$/m
      • Explaining your regex using /x
      • Using qr// and /o
    • Breakpoint: greediness and backtracking
      • What is backtracking, and how does it work?
      • What is greediness, and how do you avoid it?
      • Minimal (lazy) vs. maximal (greedy): /a\w+?z/
      • Understanding the "left-most longest" rule.
    • Substitution.
      • Using s///.
      • Calling functions with /e.
    • Simple assertions.
      • Positive look-ahead: /foo(?=bar)/
      • Negative look-ahead: /foo(?!bar)/
      • Common traps: "aaabar" =~ /a+(?!bar)/
    • More assertions.
      • Positive look-behind: /(?<=foo)bar/
      • Negative look-behind: /(?<!foo)bar/
      • The constant-width assertion plague: /(?<=ab+a)/
      • The cut assertion: /(?>a+)a/
    • Logic-flow in regexes.
      • The conditional assertion: /(?(...)a|b)/
    • Embedding code.
      • Advanced use of the /e modifier.
      • The evaluate assertion: /(?{ code })/
      • The delayed regex assertion: /(??{ regex })/
    • Advanced global searching.
      • The pos() function
      • The \G anchor and the /gc modifiers.
    • Common mistakes.
      • $&, et. al.
      • Problems with /$foo/.
      • Matching too much.
      • Wasteful modifiers.
    • Optimizing your regexes.
      • "Death to .*."
      • Match only what you want.
      • Unrolling the loop.
    • Concrete applications of regexes.
      • Data transformation for sorting.
      • Matching in reverse ("sexeger").
    • Regex tools and resources.
    Please give me your input. Is this too much for a tutorial? I want it to be in the form of Learning Perl, with exercises and such, but I want it to include as much as it should so that one has a firm grasp on Perl's regexes.

    japhy -- Perl and Regex Hacker
      I'd add "When A Regex Is Too Much", covering tr(), and "Other Places For a Regex" which would discuss split and grep.

      I use those more than substitutions and matches in plenty of programs.

        I'm definitely going to cover tr/// in the "optimization" section, showing how to change something as silly-looking as s/[aeiou]//gi to the far more competant tr/aeiouAEIOU//d.

        Don't fear; I'll be discussing functions that are oft or best employed with regexes, especially split() and its magical first argument.

        japhy -- Perl and Regex Hacker

        Edit: chipmunk 2001-05-07

      Good outline. It layers on the knowledge and complexity nicely.

      You may be thinking of covering this under "Common Mistakes".

      • Pre-match, post-match. What they do and why to avoid them.
      I think that this is a pretty good idea. It would probably teach the basics and focus on the initial learner of regexes better than anything else I've seen on the subject. Being fairly new to Perl, I would appreciate seeing something on the subject that was more advanced than the O'Reilly books, without going into the complexity of the Hip Owl book. I would be available to proof and test from a newbie perspective, if you'd like.
      Perhaps a section on regex limitations - what they can't do, problems for which they are poorly suited or simply overkill, and a good explanation of why this is so, as well as brief pointers to the right tools for these sorts of tasks (eg. Parse::RecDescent, Parse::YAPP, HTML::TreeBuilder) would also be a good idea. Best of luck =)
                     s aamecha.s a..a\u$&owag.print

      I'd love to see my oft-repeated rant addressed early on:

      • Simple patterns.
        • Plain text: /hello world/, /\Q$str\E/

              - tye (but my friends call me "Tye")
        I'll be discussing the use of regexes for simple things like /foo/. I'll mention quotemeta() and \Q...\E, of course, and suggest that something like /\Q$str\E/ be rewritten without using a regex.

        I will also discuss the dangers of /$str/.

        japhy -- Perl and Regex Hacker

      Sign me up! Kind of weird that you posted this, I was just at B&N the other day looking for the "Mastering Reg Ex" book. They didn't have that one, or anything else similar.

      Personally, I get the most out of these types of books when there are a fair number of examples and problems that allow me to test out the theory.

      Japhy: Is this too much for a tutorial?

      Personally, I think the type of person that will purchase this book will want a beefy tutorial. I know the basics of regex, but want to know them a lot better. I don't want a warm and fuzzy overview.

      Ready w/ cc in hand,


Re: japhy and mystery
by footpad (Monsignor) on May 06, 2001 at 22:03 UTC

    Well, I for one would love to see more good material on the subject. If you feel you've got the time, energy, and outline, go for it.

    That said, I would seriously think twice (or more) before commiting to a book project until you have several weeks (at least) of free time. College can be extremely demanding (depending on your major) and, unless you're as disciplined as tilly, you may find you have to scrimp the quality of your writing or the quality of your school work. Neither would be desireable.

    What I might suggest instead is that you start (if you haven't already) submitting articles to TPJ,, and other related periodicals. This will expose you to the demands of periodical publishing. While not the same as book publishing, you'll get a sense of the process, the type of people involved, and so on. These could lead to a regular column or just be an infrequent series of related articles.

    I like the idea of Perl Puzzles; perhaps that's a good hook to pursue.

    Not trying to rain on anyone's parade, just pointing out that you've got a lot on your plate at the moment. Editors are a demanding lot and really don't like it when you blow deadlines. Not that you would, but when faced with choosing between a 30-page paper worth half your grade and finishing the next three chapters in a week, one or the other will suffer. (Been there, done that.)


    P.S. And MRE's are, of course, "Meals Ready to Eat" ;-)

    Update for tilly: Well, perhaps you should be. ;-)

      There may be full-time writers who can finish a book of broad technical scope in several weeks. I don't know any who can afford to write technical books full-time, so I haven't anyone to ask. My own experience is closer to a chapter a month when I can "dedicate" myself, way less when life throws a curve or two. Remember that even if you are writing as an expert on your subject, you'll need research time, editing, diagraming and so on.

      As for that demanding lot of editors, in my experience with Manning everyone has been very supportive through a series of delays with my work. My wife developed pregnancy complications, and those deadlines went wooshing past with barely a wave from me. Maybe I'm lucky to have an editor who has kids :)

        Your experience sounds very similar to mine. I originally signed up to write DMP in 4 months. I actually took more like 15 months - and there were no complaints at all from Manning.

        Writing a book whilst holding down a full-time job is hard.


        "Perl makes the fun jobs fun
        and the boring jobs bearable" - me

      I may be many things, but disciplined is not among them...
        You mean disciplined :)

        signed Larsen, in a severe XP-whoring moment ;)

Re: japhy and mystery
by srawls (Friar) on May 06, 2001 at 23:14 UTC
    Well, I personally would love to see a book about regular expresions in perl. I see a lot of people just plain misuse them. Also, I would very much like to help you, if indeed you do decide to write a book. I could do anything you needed, from double-checking your regexes, to proof-reading your chapters, and even writing a little bit. If you need to reach me, just send an e-mail to

    The 15 year old, freshman programmer,
    Stephen Rawls
Re: japhy and mystery
by srawls (Friar) on May 07, 2001 at 01:58 UTC
    >>If you haven't been able to tell, I'm serious, and I'm a regex fanatic
    me too :)
    I liked the outline, but one thing is missing: optimization.
    UPDATE:Sorry about that, It's in there, I just missed it the first time through

    The 15 year old, freshman programmer,
    Stephen Rawls
Re: japhy and mystery
by zakzebrowski (Curate) on May 07, 2001 at 17:44 UTC
    BTW, it might be a good idea for someone who is into publishling to list a couple of pointers for people who are generally interested in writing books as well... I was pesudo thinking of writing a book on the 'over all' process of writing an application alone before the context of the language one writes in, resources used, open source vs closed source, api's (what's that?), class vs classless programing and the benefits of both... and, of course, has someone already programmed a (signifigant?) portion of the ocde you want to write...


      Well, some random pointers from my experience:

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://78369]
Approved by root
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (4)
As of 2018-07-19 17:57 GMT
Find Nodes?
    Voting Booth?
    It has been suggested to rename Perl 6 in order to boost its marketing potential. Which name would you prefer?

    Results (414 votes). Check out past polls.