http://www.perlmonks.org?node_id=790624

Apparently a previous posting of mine has been picked up by Perl-bashers. In retrospect, I shouldn't have been surprised. But I was.

The post I'm talking about proved the long-conjectured idea that Perl parsing reduces to the Halting Problem and is therefore not, in general, decidable. Stating that as "Perl is Unparseable" is a bit dramatic, but that was the terminology used in the discussion when I found it.

Unparseability is not a feature Perl folks need to be defensive about, or to minimize. Unparseability is one of the great things about Perl. OK, not unparseability in and of itself, but the ability to use the full power of the language at parse time. Because that full Turing machine power is what produces unparseability as a side effect.

Compile-phase processing is heavily used in Perl, and having the full language available at that point is one of its major advantages over other languages. That's why nobody ever talks about "fixing" Perl's unparseability. It's not a problem and doesn't need a fix.

With the usefulness of having the full language in the compile phase, contrast the unparseability downside: yes, there are potential Perl programs that cannot be parsed. None I've ever wanted to actually use, mind you. The only Perl programs I've encountered that don't parse are the artificial ones invented for the discussion of unparseability.

So the trade-off is practical power for theoretical purity. Thank you, Larry Wall. And, hey, designers of other languages! How come you didn't make such a smart trade-off?

So, monks, next time you hear Perl jeered at for having undecidable parsing, just tell 'em -- demanding a parseable language is the sign of weak programmer.

[ My proof is available in a series of three articles in The Perl Review, now available online. The TPR articles should be preferred to my original post because they are more fully explained, were carefully rewritten, and benefited from editing by brian d foy. ]

Replies are listed 'Best First'.
Re: Unparseability is A Good Thing
by zby (Vicar) on Aug 23, 2009 at 13:30 UTC
    Hmm - but isn't there any way to have both "the ability to use the full power of the language at parse time" and still be parseable? If we removed just subroutine prototypes rom the language the proof would not hold any more. So the question is is there a more general proof that does not rely on prototypes?
[reply]

      If we removed just subroutine prototypes rom the language the proof would not hold any more. So the question is is there a more general proof that does not rely on prototypes?

      Yes: 

      BEGIN {
   eval "sub foo {}" if rand() < 0.5;
}

foo 'x';

      [download]
      >perl -c test.pl
test.pl syntax OK

>perl -c test.pl
String found where operator expected at test.pl line 5, near "foo 'x'"
        (Do you need to predeclare foo?)
syntax error at test.pl line 5, near "foo 'x'"
test.pl had compilation errors.

      [download]

      Execution is required to decide whether foo is a subroutine call or a bareword.

[reply]
[d/l]
[select]
      Yet another proof: 
      BEGIN {
   my ($add_ts) = @ARGV;
   eval 'use subs qw( die )' if $add_ts;
}

sub die {
   my $ts = '[' . localtime() . '] ';
   my $msg = join '', @_;
   $msg =~ s/^/$ts/mg;
   CORE::die($msg);
}

die("foo\n");

      [download]
      >perl test.pl 1
[Sun Aug 23 11:52:55 2009] foo

>perl test.pl 0
foo

      [download]

      Execution is required to decide whether die is a subroutine call or an operator.

[reply]
[d/l]
[select]
        I don't agree with this one. An interpreted language can be parsed, by definition. To say something cannot be 'parsed' is to say the program itself is non-deterministic in its syntax (not its outcome).

        If I remember where this bandwagon came from, it was someone's project who realized the project was too hard to write perl to run perl. It was not proven that it cannot be done, however often touted.
[reply]
      The entire language is available at compile time (via BEGIN blocks). I think that this fact makes any distinction between "parsing perl" and "executing perl" a little artificial & disingenuous in the context of this undecidability proof.

      Anyway, since the entire language is available in BEGIN blocks, the proof does not crucially rely on subroutine prototypes. They just provide a convenient "parsing property" (clearly, determining the prototype of a sub is something that should result from "parsing") in which to effect the result. Any aspect of the language that you want to call a "parsing property" can be used. One that comes to mind is whether a loaded module returns a true value or not (and therefore whether the "parse" succeeds) -- this is something that happens during BEGIN phase.

      blokhead

[reply]
Re: Unparseability is A Good Thing
by jdrago_999 (Hermit) on Aug 24, 2009 at 17:55 UTC

    Thanks for putting into words one of Perl's most special traits. A "mutant skill" of Perl.

    One of my favorites: 

    package Foo;

# Some kind of just-in-time configuration happens here...
sub init {
  push @INC, '/my/module/dir';
}

    [download]

    Elsewhere... 

    package Bar;

use Foo;
BEGIN { Foo->init }
use Baz; # Resides under /my/module/dir

    [download]
[reply]
[d/l]
[select]
Re: Unparseability is A Good Thing
by papidave (Pilgrim) on Aug 26, 2009 at 21:37 UTC
    Amen. Kudos to LW for choosing function over form for the few cases where we couldn't have both. I'm not entirely certain that "undecidable" from the original node equates to "unparseable" in this one, but anyways...

    Isn't FORTRAN undecidable? F66 isn't pretty, but it has certainly proved itself useful over the last 4+ decades.

    I had considered doing a full analysis of the proof, but I'm not sure if/when my brain would ever complete that task. I guess it's Just Another Perl Halting_Problem.

[reply]

Back to Meditations