Filter::Simple bug with here documents

Though I've already reported this bug, along with a fix, I have no idea how the fix works. I'm completely mystified. Here's a small test case:

package NoFilter;

use Filter::Simple;

FILTER_ONLY code => sub {};

1;
[download]

In other words, it filters nothing.

Here's my test code:

#!/usr/bin/perl

use NoFilter;

my $buf = <<HERE;
/
HERE

my $buf2 = <<HERE;
HERE
[download]

This generates the error:

substr outside of string at /usr/local/lib/perl5/5.8.7/Text/Balanced.pm line 70.

At first blush it looks like Text::Balanced has a problem with the HERE docs (and historically it has had problems with HERE docs). However, there's a relatively easy fix for this. Opening up Filter::Simple and adding [$_] as the first line of the anonymous subroutine which Filter::Simple::gen_std_filter_for returns:

sub gen_std_filter_for {
    my ($type, $transform) = @_;
    return sub {
        [$_];   # XXX this is the "voodoo" line
        my $instr;
[download]

And the problem mysteriously goes away. I have no frickin' idea why. Any clues? I've emailed thedamian about this but frankly, that's one heck of an embarrassing bug fix since it's "voodoo". Is there something bizarelly simple that I'm overlooking?

Side note: you can also have fun by deleting the "/" line in the first here doc. That generates other fun error messages.

This is perl, v5.8.7 built for darwin-2level. I'm using the latest Text::Balanced and Filter::Simple. I can replicate this problem on OS X and Fedora Core 2. A friend of mine can't replicate this on Perl 5.8.5 Redhat 4.0.

Cheers,
Ovid

New address of my CGI Course.

Comment on Filter::Simple bug with here documents Select or Download Code
Re: Filter::Simple bug with here documents by GrandFather (Cardinal) on Dec 02, 2005 at 05:01 UTC
Curiosity only but, what lead you to try `[$_];` there? DWIM is Perl's answer to Gödel	[reply] [d/l]
Re^2: Filter::Simple bug with here documents by Ovid (Cardinal) on Dec 02, 2005 at 05:09 UTC
What? You don't just randomly insert `[$_]` throughout your code to see what happens? :) After much debugger work (and using the debugger with source filters is not fun), I found the error appeared to be coming from `&Text::Balanced::extract_multiple`. Since I couldn't get it to generate the error directly, I fell back to using an empty source filter and that kicked up the error. `&Filter::Simple::gen_std_filter_for` is the only place in Filter::Simple where extract multiple is called: `sub gen_std_filter_for { my ($type, $transform) = @_; return sub { my $instr; local @components; for (extract_multiple($_,$extractor_for{$type})) {` [download] So naturally, right before that line I added this: `use Data::Dumper; warn Dumper( [ $_, $extractor_for{$type} ] );` [download] At that point, the error went away, I sneezed and my head imploded (how Britney Spear's head fails to implode when she sneezes is another mystery I haven't solved). More playing around got that down to the "fix" which I posted. Cheers, Ovid New address of my CGI Course.	[reply] [d/l] [select]
Re: Filter::Simple bug with here documents (duh!) by tye (Cardinal) on Dec 02, 2005 at 06:10 UTC
Don't do that! FILTER_ONLY will always have bugs. Just don't use it! It is really just that simple. And the problem mysteriously goes away. Yes. Didn't you learn anything from people's attempt to use Switch.pm?? Things mysteriously break (and mysteriously unbreak). Just STOP IT! You actually can write reasonable source filters for Perl. You just need to avoid trying to not have your filter apply to quoted things (or other subsets of Perl syntax). So define your own way to escape things for places when the filter would be a pain. And define your custom syntax so you don't need to escape away from it much. And never, never use FILTER_ONLY! - tye	[reply]
Re^2: Filter::Simple bug with here documents (duh!) by Ovid (Cardinal) on Dec 02, 2005 at 06:13 UTC
Can you provide some context for these dire warnings? Cheers, Ovid New address of my CGI Course.	[reply]
Re^3: Filter::Simple bug with here documents (duh!) by tye (Cardinal) on Dec 03, 2005 at 03:28 UTC
FILTER_ONLY tries to parse Perl code. Did you really not realize this or not realize that such is doomed and thus something quite foolish to include in production code? - tye	[reply]
Re^4: Filter::Simple bug with here documents (duh!) by Ovid (Cardinal) on Dec 03, 2005 at 04:11 UTC
Re: Filter::Simple bug with here documents by nexion (Initiate) on Sep 13, 2011 at 18:39 UTC
This seems to be an issue with 5.8. I ran in to this issue myself on 5.8 and found that 5.10, at least on my system, does not trigger the bug.	[reply]


The stupid question is the question not asked
	PerlMonks

Filter::Simple bug with here documents

Don't do that!

And never, never use FILTER_ONLY!

How many continents have you visited?