Re: Applying regexes to streams: Perl enhancement idea

[Juerd]

I hope this can make it into Perl 6 regexes

exegesis 5:

The inability to do pattern matches immediately on an input stream is one of Perl 5's few weaknesses when it comes to text processing. Sure, we can read line-by-line and apply pattern matching to each line, but trying to match a construct that may be laid out across an unknown number of lines is just painful.

Not in Perl 6 though. In Perl 6, we can bind an input stream to a scalar variable (i.e. like a Perl 5 tied variable) and then just match on the characters in that stream as if they were already in memory

(...)

The important point is that, after the match, only those characters that the pattern actually matched will have been removed from the input stream.

It seems that your wish will come true.

# Perl 6
my $variable is from($stream);

- Yes, I reinvent wheels.
- Spam: Visit eurotraQ.

[tye]

The important point is that, after the match, only those characters that the pattern actually matched will have been removed from the input stream.

In the more general case, what is removed from the input stream is the characters that actually matched and any characters prior to them.

But note that this still leaves the final problem I outlined. It is extremely memory-inefficient when matching short strings that occur very infrequently in very long streams.

Well, the description is vague enough that perhaps there are also already plans to support such cases more efficiently, though I'm skeptical of that.

Also, by the time I finished writing this up I had decided that patching this into Perl 5 might be quite easy. So I still have a couple of wishes left.

Thanks for the pointer.

                - tye