Re: RFC: Inline::Blocks or inline as a keyword?

[holli]

Source filtering? No!

You seem to have missed Damians latest(?) stroke of genius resulting in Keyword::Declare and PPR.

[shmem]

Source filtering? No!

Quick, throw away your C pre-processor! and along with that, all that autoconf and automake rubble.
Seriously, why should source filtering be bad for perl, but good for C?

I have explained in the op why in this case source filtering makes sense, and I see no point against that if it is done sensibly. If I don't do it sensibly, please point out why.

You seem to have missed Damians latest(?) stroke of genius resulting in Keyword::Declare and PPR.

I missed them only slightly. I am aware of those beasts, but I didn't delve much into them because I currently haven't any use for them. Please don't compare me to Damian, that's unfair to both of us.

[holli]

I think there's a misunderstanding here, I am not comparing anything. I wanted to express that the Keyword::Declare module is the tool you need to implement your module without source filters or rolling your own perl parsing regexes.

[shmem]

Thanks for clarifying, and apologies for my misunderstanding, which stems from:

Source filtering? No!

Since - in what way is Keyword::Declare not a source filter? From the description of that module (emphasis mine):

At compile time, when the parser subsequently encounters source code such as:

loop 10 {
    $cmd = readline;
    last if valid_cmd($cmd);
}

then the keyword's $count parameter would be assigned the value "10" and its $code parameter would be assigned the value "{\n$cmd = readline;\nlast if valid_cmd($cmd);\n}". Then the "body" of the keyword definition would be executed and its return value would be used as the replacement source code:

for (1..10) {
    $cmd = readline;
    last if valid_cmd($cmd);
}

Skimming that module, I get the impression that it is definitely a source filtering module. It is not a general purpose macro module. It is not a module for inlining subroutine bodies at the calling places as do blocks. It implements a "domain-specific language" (this might be the wrong term, beat me to death for it please) to extend Perl with arbitrary keywords. Right?

That's all ok, and I admire Damian's genius.

But the purpose of this meditation was not about "the right way to implement" inlining, but rather asking (see title) - should we have an inline keyword, or have a module for that. The attached module is a POC, a thinly veiled attempt to get approval and XP, and something which just does what it claims to do, nothing else.

Had there been written instead

A more robust way to implement your module is Keyword::Declare, since (arguments here)

no misunderstanding would have arisen. Instead in the follow-up I read

I wanted to express that the Keyword::Declare module is the tool you need to implement your module without source filters or rolling your own perl parsing regexes.

which induces violence phantasies in me and, at the same time, drops me into utter resignation. Gosh, what a shitty job did I do. No wonder, cuz there will always be someone brighter than me, and they can annihilate my efforts with just one fell stroke of genius. Then there is the the tool you need to implement your module which, as so often, means "do as I say, but I won't tell you how." And it sounds so pythonesque. There's one way. Leaving away the "source filters" part, since that was discussed above.

I'm so fed up with, and consider harmful, such advises as

Q: How can I match IPv4 addresses reliably? I use /\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}/
A: Don't. use Regexp::Common

without at least saying "if you want to know what else you have to check, read the source" or at least saying that 377 isn't a valid dotted quad component and why.

So, somebody please (if it is worth the effort) implement this module in a more robust fashion using Keyword::Declare.

Lastly, to address the rolling your own perl parsing regexes - yeah, that's the right way to go, don't roll your own, blame somebody else. In one of my first posts here I answered to don't re-invent the wheel with "I don't want to reinvent the wheel, I just want a wheel with no car around just in case I want to build a nifty chopper".

No more choppers, I'm tired of it all. Bye.

[RonW]

Despite such vehement discouragement, there's nothing (IMHO) intrinsically wrong with source filtering. It's just requires being very careful when using it. Also, it can make a program it using harder to debug.

Anyway, I can suggest how you might use Keyword::Declare to accomplish your inlining module. Though with a modified syntax.

Although Keyword::Declare will let you redefine sub (and other keywords), it won't do what you want. Instead, I suggest adding another keyword, maybe inlinable to mark a subroutine as suitable for inlining. Your handler for inlinable would save the body of the sub for later use by inline, then return the sub definition without the inlinable keyword.

For inline sub definitions, save the body, then instead of returning the sub definition, use the name to define a new keyword. When perl later encounters the name of the sub, it will be treated as a plug-able keyword. then your handler can inline the routine.

Caveat: Keywords defined by Keyword::Declare are only recognized at the beginning of a statement. If you want to inline in the middle of a statement, you have to enclose the "call" in a do { } block.

There might be a plus side to this caveat. Since you know the inlining will always take place at the start of a statement, you might not have to inline it as a do { } block, but as a plain block. This is because if your inlined code isn't "returning" a value, you shouldn't need the do { } block. But if it does "return" a value and you want that value, the "call" will already be inside a do { } block (as long as it is the last (or only) statement in the do { } block).

Summary: A Keyword::Declare based module may be easier to fully implement, but a source filter version probably will result in a cleaner syntax. The Keyword::API might be able to install keywords that work in the middle of expressions, but is not as easy to use as Keyword::Declare.

Disclaimer: I have not tested any of this. YMMV.

Edited to correct grammer errors.

[Eily]

Seriously, why should source filtering be bad for perl, but good for C?

In both cases the compiler might report an error where the mistake was inserted, rather than where it actually is. Also in perl, source filters screw up the line numbering (insert two lines in the code, and the line numbers for every compilation message, or warn or die afterwards will be off by two). A call to warn on the last line of your test will indicate an issue on line 35 of a 26 lines file...

The #line directive can solve most of those issues though, so if you turn

sub capitalize_next {
    my ($thing) = @_;
    uc inline increase($thing);
}
sub increase {
    my ($foo) = @_;
    ++$foo;
}

into

sub capitalize_next {
    my ($thing) = @_;
    uc do { local @_ = ($thing);
#line 6
      my ($foo) = @_;
      ++$foo;
#line 3
    };
}
sub increase {
    my ($foo) = @_;
    ++$foo;
}

Mistakes in the parameter list (eg increase(£thing)) will be marked as coming from line 3 (the line of the call), and mistakes from the function definition (eg ++£foo) will be reported as coming from inside the function definition.

local @_ makes it possible to access $_[0] and other values inside @_, while making line numbering easier (if ($foo) = @_ is replaced by ($foo) = $thing, the LHS comes from the function body but the RHS from the call, so where do you report an erreor?), but it probably slows things down... And since $foo = shift; uses @_ inside a function, but @ARGV elsewhere, it might introduce some interresting bugs.

Also note that the C preprocessor will only expand macros in code, not in comments nor in strings. Try this in your example:

sub capitalize_next {
    my ($thing) = @_;
    # uc inline increase($thing);
}

and you'll get some confusing errors. It's pretty easy to correct that one, but things might get difficult with strings and pod.

[shmem]

In both cases the compiler might report an error where the mistake was inserted, rather than where it actually is.

Well, after the file is filtered and text inserted via inline directive expansion, the erroneous code is duplicated and defacto is actually also at the new location. That's how inlining works. Anybody using a source filtering module should know what they are doing. The confusing line numbers issue can be mitigated by running -MO::Deparse,-l on the faulty file, and the line numbers reported by the compiler will match those of the output.

This is precisely why I included the debug switch into the module configuration, which causes the output of the code after mangling. But yes, there are much more CAVEATS than those currently mentioned in that section.

Thanks for your input!