Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses

Filter::Simple bug with here documents

by Ovid (Cardinal)
on Dec 02, 2005 at 04:51 UTC ( #513511=perlquestion: print w/ replies, xml ) Need Help??
Ovid has asked for the wisdom of the Perl Monks concerning the following question:

Though I've already reported this bug, along with a fix, I have no idea how the fix works. I'm completely mystified. Here's a small test case:

package NoFilter; use Filter::Simple; FILTER_ONLY code => sub {}; 1;

In other words, it filters nothing.

Here's my test code:

#!/usr/bin/perl use NoFilter; my $buf = <<HERE; / HERE my $buf2 = <<HERE; HERE

This generates the error:

substr outside of string at /usr/local/lib/perl5/5.8.7/Text/ line 70.

At first blush it looks like Text::Balanced has a problem with the HERE docs (and historically it has had problems with HERE docs). However, there's a relatively easy fix for this. Opening up Filter::Simple and adding [$_] as the first line of the anonymous subroutine which Filter::Simple::gen_std_filter_for returns:

sub gen_std_filter_for { my ($type, $transform) = @_; return sub { [$_]; # XXX this is the "voodoo" line my $instr;

And the problem mysteriously goes away. I have no frickin' idea why. Any clues? I've emailed thedamian about this but frankly, that's one heck of an embarrassing bug fix since it's "voodoo". Is there something bizarelly simple that I'm overlooking?

Side note: you can also have fun by deleting the "/" line in the first here doc. That generates other fun error messages.

This is perl, v5.8.7 built for darwin-2level. I'm using the latest Text::Balanced and Filter::Simple. I can replicate this problem on OS X and Fedora Core 2. A friend of mine can't replicate this on Perl 5.8.5 Redhat 4.0.


New address of my CGI Course.

Comment on Filter::Simple bug with here documents
Select or Download Code
Re: Filter::Simple bug with here documents
by GrandFather (Cardinal) on Dec 02, 2005 at 05:01 UTC

    Curiosity only but, what lead you to try [$_]; there?

    DWIM is Perl's answer to Gödel

      What? You don't just randomly insert [$_] throughout your code to see what happens? :)

      After much debugger work (and using the debugger with source filters is not fun), I found the error appeared to be coming from &Text::Balanced::extract_multiple. Since I couldn't get it to generate the error directly, I fell back to using an empty source filter and that kicked up the error. &Filter::Simple::gen_std_filter_for is the only place in Filter::Simple where extract multiple is called:

      sub gen_std_filter_for { my ($type, $transform) = @_; return sub { my $instr; local @components; for (extract_multiple($_,$extractor_for{$type})) {

      So naturally, right before that line I added this:

      use Data::Dumper; warn Dumper( [ $_, $extractor_for{$type} ] );

      At that point, the error went away, I sneezed and my head imploded (how Britney Spear's head fails to implode when she sneezes is another mystery I haven't solved). More playing around got that down to the "fix" which I posted.


      New address of my CGI Course.

Re: Filter::Simple bug with here documents (duh!)
by tye (Cardinal) on Dec 02, 2005 at 06:10 UTC

    Don't do that!

    FILTER_ONLY will always have bugs. Just don't use it! It is really just that simple.

    And the problem mysteriously goes away.

    Yes. Didn't you learn anything from people's attempt to use Things mysteriously break (and mysteriously unbreak). Just STOP IT!

    You actually can write reasonable source filters for Perl. You just need to avoid trying to not have your filter apply to quoted things (or other subsets of Perl syntax). So define your own way to escape things for places when the filter would be a pain. And define your custom syntax so you don't need to escape away from it much.

    And never, never use FILTER_ONLY!

    - tye        

      Can you provide some context for these dire warnings?


      New address of my CGI Course.

        FILTER_ONLY tries to parse Perl code. Did you really not realize this or not realize that such is doomed and thus something quite foolish to include in production code?

        - tye        

Re: Filter::Simple bug with here documents
by nexion (Initiate) on Sep 13, 2011 at 18:39 UTC
    This seems to be an issue with 5.8. I ran in to this issue myself on 5.8 and found that 5.10, at least on my system, does not trigger the bug.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://513511]
Approved by bobf
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (4)
As of 2014-09-20 04:33 GMT
Find Nodes?
    Voting Booth?

    How do you remember the number of days in each month?

    Results (152 votes), past polls