Experimenting with Lvalue Subs

[Limbic~Region]

All,

Lvalue subs are experimental. It says so right in perldoc perlsub. They have been experimental since their introduction in 1999, nearly 6 years ago. Everytime someone asks how to do argument validation with lvaluable subs, the mantra is "lvaluable subs are experimental and shouldn't be used, there are modules that likely do what you want already, but if you must do that then you will need to tie the return variable which is going to make it slow".

While not exactly the same, this is similar to the response about using map in a void context. Readability issues aside, the efficiency problem turned out to be a 3 line patch. I feel like the same thing is happening with Lvalue subs and am interested in what you think. Consider the following snippet:

# Somewhere inside the object's package
sub get_set :lvalue {
    my ($self, $key) = @_;
    $self->{ $key };
}

# Your code that uses it
my $obj = Some::Object->new();
$obj->get_set( 'name' ) = 'Limbic';
print "Hello, ", $obj->get_set( 'name' ), "\n";

It allows me to use a single accessor/mutator method. Of course, the same thing could be done as
$obj->get_set('name' => 'Limbic')
but some people really prefer the assignment notation. So what's the problem? Well, first I might want to restrict what keys can actually be set. Borrowing a trick from Zaxo, you can do this with the current Lvalue implementation:

sub get_set :lvalue {
    my ($self, $key) = @_;
    my $junk;
    key_ok( $key ) ? $self->{ $key } : $junk;
}

While this will silently doesn't do what you asked, it could be made to be verbose but it still doesn't allow you to validate the rvalue. It isn't on the @_ stack. In fact, as far as I know, there is no way to get to the rvalue inside an Lvalue sub. Have given this some thought, discussing it in the CB, thrown out some possible implementations, I think there are only a couple of things that are necessary to make Lvalue subs more useable.

• A way to tell if it is being used in assignment or not
• When in assignment, a way to get at the rvalue(s)

Of course there are a few other things that might make them nicer. A straight forward alternative to using a $junk variable and ternary to avoid assignment if validation fails would be nice. I would imagine that you could manipulate the rvalue(s) prior to the assignment, but it might also be nice to have pre and post hooks for syntatic sugar purposes.

I am not saying that these changes would only be 3 lines but is that any reason not to make them more useable? When I mentioned in the CB that the problem was having a competent C programmer write the patch and then having the fortitude to fight for it on p5p. The response from a monk who has been around the Perl community for quite some time and has been involved with p5p indicated the real problem was the latter not the former. I find this hard to understand. If there is no currently promised behavior and someone provides a working patch to make it useable, why not apply it?

So what do you think - about extending Lvalue subs at all, about the difficulty in doing so both from a C perspective as well as getting the patch applied, about what features and implementation would really make it worth the effort?

[dragonchild]

First, I think that anything that adds to a given feature in Perl, whether I use it or not, is a good thing. I like your ideas and would love to see the discussion go further. That said, I have a pretty good idea of why p5p might be ... reluctant to do more with lvalue subs.

I maintain a webapp that's half in Perl, half in Javascript. It's almost 8 years old and there have been at least 15 different people who've worked on it, then walked away. There are no more features for this thing - just bug fixes. And, each bugfix, no matter how trivial, takes at least a week from start to finish. I just wrote up a proposal to fix a bug that affects one aspect of one feature when it's used with another feature. This fix will take at least 3 months and involve testing the entire application. Reason - the bug is in a function that every feature in the application uses.

Perl is kinda like that. It's not a clean, well-factored application. The guts are a mess of C macros, typedefs, crazy optimizations, and OS-specific code. When you start messing with something as integral as subroutines, you never have enough regression tests. You're talking about affecting a core subsystem that, probably, many other subsystems depend on in some way or another. And, it may not be clear what depends on it. I've fixed bugs in section A that exposed hidden dependencies in section G.

However, despite all that, I think the discussion is definitely a good one. Without a lance, you cannot tilt at windmills.

[TimToady]

It's fine by me if you want to do this, as long as you do it the same way Perl 6 is going to do it—or close enough that the translator won't have issues. That means, among other things, that the body always runs, even in lvalue context, and any validation is done by some kind of a callback, not by the body code, which is responsible *only* for identifying and returning some object that can proxy for the attribute in question.

[TheDamian]

Specifically, see: Apocalypse 6

[theorbtwo]

You know, when I first read that section of A6, I thought "that's as ugly as anything I've ever seen... eh, I'm sure it'll get changed". Now, around 3 years later, I re-read it, and I think "that's as ugly as anything I've ever seen -- it's gotta get fixed, or we'll be stuck with it". This doesn't match how I think about lvalue subs. It doesn't match how I think of lvalue subs. I think of lvalue subs as a different calling convention from a normal sub -- a view reinforced by that being the way most languages that have them seem to think of them. The way you're doing it seems to match the way perl5 does it. Lvalue subs aren't much used in perl5, and them being annoying to use is one reason for that.

Why not make $foo->bar('baz')='quux'; with a declared method multimethod bar($this: *@lvalues, @rvales :rvalues); does... well, the obvious: makes $this be $foo, @rvalues be ('bar') and @lvalues be ('quux'). IOW, make the lvalues be a special zone (in A6 terms), with no presigil signifier, but only a long name. (I suppose you haven't yet used > (points to the left) for anything in this context, but it'd be confusing, because it comes up rarely, and could be difficult to find in the docs, whereas "rvalue" is somewhat obvious, or at least easier to look up.)

As is normal for multimethods, that body is only called if the call matches it -- that is, it's done in an lvalue context.

One difficulty is in determining the context of the RHS of the assignment. Normally, the context of an assignment is determined by what is being assigned to. This is one of the major reasons in both p5 and p6, the lvalue is evaluated before the rvalue, which is why the lvalue sub can't tell what rvalue is being assigned. This needs an aditional rule, I think, instead of a workaround that keeps information from the programmer when they want it. If the lvalue sub has only a single scalar paramter in the :rvalues zone, then it's in scalar context. Otherwise, it's in list context. (If there are two multimethods that have equally compatable signatures, and one of them would be list context by this rule, and the other scalar, then warn, and do it in list.)

[TimToady]

The purpose of an lvalue sub is to return an lvalue.

[theorbtwo]

As I look more, I begin to see a problem in my plan. It's in the last paragraph of the above, which assumes that we can figure out which of a set of multimethods will be called in any purticular circumstance very early -- it'd require evaluation of the RHS before we determine what context to evaluate the RHS in. It may be neccessary to provide the context of the RHS as an explicit attribute of the method, and disallow having different versions that vary by rvalue context with the same non-rvalue signature.

The order of evaluation would then go as follows:

1. Invocant.
2. LHS arguments
3. At this point, we determine which of the potentially competeing multimethods to call. If there are none, or more then one with the same goodness, error (possibly not error, but just pick the first or last in the second case).
4. Determine the context for the RHS.
5. Evalute the RHS.
6. Finally, call the actual lvalue method.

I realize that this makes reverting the value somewhat more complex from the core -- but if the core thinks of the rollback as being a new assignment, and propigates exceptions thrown, it shouldn't be much more difficult. (Note that this is hardly the only place that exceptions can be thrown from nonobvious places.) I think it's far more important to give an interface to lvalue subs that matches the way people think about them.

[Zaxo]

You missed one part of the trick. The body of an lvalue sub doesn't get executed when it is used as an lvalue. The trick works only if the last expression is an lvalue (as is trinary op). That means you do no good to shift in or copy the function arguments. $key and $self in your example are undefined when called as an lvalue.

Arguments are still usable, but you need to initialize them in the last line of the sub. Here's a minimal rewrite.

sub get_set :lvalue {
    my $junk;
    key_ok( $_[1] ) ? $_[0]->{ $_[1] } : $junk;
}

Note that $junk is effectively a package global variable holding the last value assigned with a not-ok $key. You can retrieve it with $foo->get_set($bad_key), but the value returned is for any $foo of the class.

[Limbic~Region]

Zaxo,
The body of an lvalue sub doesn't get executed when it is used as an lvalue.

Not sure if that was the original behavior back in 1999 when they were first introduced but newer versions certainly disagree with you:

my %foo;
get_set( 'foo' ) = 42;
print get_set( 'foo' );

sub get_set :lvalue {
    my $key = shift;
    print "key is $key\n";
    my $junk;
    $key eq 'foo' ? $foo{$key} : $junk;
}
__END__
key is foo
key is foo
42

...You can retrieve it with $foo->get_set($bad_key)

Where $bad_key is any and all bad keys. This is why one of the desired features would be to determine if it was in assignment or not.

[Zaxo]

Now _that_ is interesting. Anybody know when it was introduced?

/me is off to experiment anew . . .

[BrowserUk]

I totally agree, and said as much in What the [sub]in 'L value do they have?. I even suggested a few ways that the value could be made available to the sub without breaking existing code:

1. By placing it into the scalar slot associated of the typeglob of the functions name.
2. By placing it into $^_.

   This is analogous to $^N as used in the (??{ code }) regex construct and is mnemonic* with $_.

   Corrected per dragonchild's post below.

   It could also be extended to @^_ for lvalue list assignment--which has interesting possibilities.

I think it probably would just require someone with the tuits to make such modifications to the sourcecode to produce the patch.

[dragonchild]

By placing it into the scalar slot associated of the typeglob of the functions name.

That's not so good - you run the risk of clobbering something. Doing an implicit 'local' might help, but what do you do with lvalue-able anonymous subs?

*^_ is a much better idea, imho. And, I'd just have it go to @^_ straight off. That way, it would act as a standard assignment does. $foo = 1..3; anyone?

(FYI - it's mnemonic, not pnemonic ... you want memory, not the plague.)

[BrowserUk]

...but what do you do with lvalue-able anonymous subs?

Good point. Then I agree, @^_ is best.

[fergal]

I thought about that too but one problem is order of evaluation. In order to provide the rvalue to the sub, you would have to evaluate the right hand side before the left hand side and that's odd or possibly even bad (I'm not exactly sure yet).

I've also seen Larry Wall say somewhere that tieing is the way you will have to do this in Perl 6 (although it will be considerably more straight forward that in Perl 5)

[BrowserUk]

...you would have to evaluate the right hand side before the left hand side...

I'm not sure that I understand you. The right-hand side of an assignment is always evaluated before the assignment can occur? Except maybe in the case of assigning a constant, but even then the value of that constant is fetched from wherever before it can be assigned to the lvalue.

It's quite possible that I do not fully grasp the purpose or magic that lies behind the implementation of lvalue subs, but any code in an lvalue sub is invoked as a sideeffect of the assignment to an lvalue sub:

{ my $x; sub x: lvalue { print 'fred'; $x } };
x = 123;
fred

The problem is that you can't do anything useful with that code as you cannot affect the outcome of the assignment relative to the value being assigned.

[Anonymous Monk]

By placing it into the scalar slot associated of the typeglob of the functions name.

You can't. It has already been taken by AUTOLOAD.

[fergal]

Have a look at Class::Accessor::Lvalue and a discussion of Lvalue accessors, in which you can find my own slightly different take on it, which I might put on CPAN.

Just on what you've written here, I think

$obj->get_set( 'name' ) = 'Limbic';

is not a great idea as it gives you many of the problems of direct hash access. It seems to me that creating individual Lvalue methods for each field in you object gives you run time checking without having to write a key_ok() sub and it also gives you overridability.

$obj->name = 'Limbic';

seem much nicer to me (although I prefer fields to have an initial capital letter).

As for getting the rvalue, it's possible but it's not pretty! Basically you use a tied scalar as the lvalue and then when it gets assigned into, you can see the rvalue and you can do whatever tricks you like. In my own class and also in C::A::L when the tied object is written to or read from it just calls the setter or getter method respectively. This gives you full overridability, just override the default setter method.

There's lots of overhead but you can do cool stuff like a two-way hash, for example:

$dept->Positions("manager") = "bill";
print $dept->People("bill")."\n";
#  manager

here we have overridden setPositions, getPositions, setPeople and getPeople so that each of the setters updates 2 hashes internally, 1 mapping from People to Positions and the other mapping the opposite direction. The getters just read from the relevant hash.

2005-01-29 Janitored by Arunbear - fixed hard-coded pm link, as per Monastery guidelines

[Limbic~Region]

fergal,
Have a look at...

*Chuckle* One of the first points I made was that instead of fixing the implementation of Lvalue subs, people chant the mantra:

• 1. Don't do that
• 2. If you are going to do that, use a ready made wheel
• 3. If you insist doing it yourself, you will need to tie it which makes it S-L-O-W

If we are going to revert to other wheels, I personally prefer Attribute::Property and even discussed how to overcome some of its limitations. It doesn't fix the fact that a core feature is pretty much unuseable in the current form.

As for getting the rvalue, it's possible but it's not pretty!
Not really. I already mentioned that tying is the last ditch effort in the mantra and I am well aware of how to do it. The trouble is that you are not getting at the rvalue inside the sub which makes tying necessary. The point of the meditation is to spark interest in correcting this deficiency, not more work arounds.

Unfortunately I am not a competent C programmer, but TimToady indicated elsewhere in this thread that he wasn't opposed to the change as long as it was in alignment with p6. And all this time I was assuming the trouble would be with getting it past p5p not the patch itself.

[fergal]

Indeed, sorry for not reading more carefully. However my comment isn't entirely void, have a look at Apocalypse 6 where Larry says that lvalue subs will not get to see their rvalue unless they use tieing

If you need to do pre- or post-processing on the "public" value, however, you'll need to return a tied proxy variable.

and he also said that the tieing will be easy. He has other plans (temporizing - basically a tie interface to local) that would break if it was done the way you want.

There's still also the issue of potentially surprising order of evaluation.

[runrig]

Can you accomplish lvalue validation with Want's LVALUE and ASSIGN specifiers?

[Limbic~Region]

runrig,
I just spent the better part of an hour playing with Want. Indeed, it can do what I was suggesting albeit very clumsily.

#!/usr/bin/perl
use strict;
use warnings;
use Want;

my %obj;
sub cool :lvalue {
    my $key = shift;
    my %ok = map { $_ => 1 } 1..7;
    my $junk;

    if ( want( 'LVALUE ASSIGN' ) ) {
        if ( ! $ok{ $key } ) {
            warn "$key is not valid";
            lnoreturn;
        }
        else {
            my ($rval) = want( 'ASSIGN' );
            if ( ! $ok{ $rval } ) {
                warn "$rval as rval is not valid";
                lnoreturn;
            }
            $obj{ $key } = $rval;
        }
        lnoreturn;
    }
    elsif ( want( 'RVALUE' ) ) {
        if ( ! $ok{ $key } ) {
            warn "$key is not valid";
            rreturn $junk;
        }
        else {
            rreturn $obj{ $key };
        }
    }
    else {
      warn "Houston, we have a problem";
    }
    return;
}
cool( 1 ) = 3;
print cool( 1 ), "\n";
cool( 5 ) = 42;
cool( 9 ) = 3;
print "$_ : $obj{$_}\n" for keys %obj;
__END__
3
42 as rval is not valid at foo.pl line 20.
9 is not valid at foo.pl line 14.
1 : 3

At first I thought it must be a source filter then I noticed that it used XS and stopped looking. It is neat but not practical IMO.

[runrig]

To me the only clumsy thing is that you have to supply those ugly arguments to the want() function. It might be nicer to have something like is_rvalue() and is_lvalue() aliases to get to that functionality. I'm not sure what else you could do to make it less clumsy, did you have anything else in mind?

[ysth]

I wouldn't mind seeing :lvalue subs improved; but I'm not seeing clear suggestions here for how to do so (except ones that only take into account use in a scalar assignment). Any suggested improvement should
• not make list-context :lvalue subs work any worse than they do now :)
• specify what happens in more complicated cases:

  sub foo:lvalue { ... }
  (undef,foo)=bar();
  bar(foo);
  bar(\foo);
  foo ||= bar;
  bar(foo()++);
  

Given that, I don't see how the sub body itself can meaningfully have access to the rvalue, which IMO leads directly to TimToady's position that the sub body's job is only to return an lvalue, and that the room for improvement lies in syntax for specifying callbacks.

[BrowserUk]

What you're saying is that providing the rvalue to the sub for validation would require the sub to be called twice. Once to obtain (a reference to) the lvalue, and the second time to allow the verification.

But if the mechanism to validate is to tie the lvalue, then a second call is going to be made anyway--to the STORE method of the tie. No different!

Except that you now have a piece of out-of-band code performing the validation, and second level of lookup to locate the appropriate tie class and a third level of lookup locate the STORE method within that.

And all this to avoid a second call in the rare case when the assignment will be undone at the end of scope?

[ysth]

I'm not sure what you are arguing for or against; I'm saying the primary purpose of an :lvalue sub is to create an lvalue, and that there are many cases where the creation of the lvalue is separated in time and space from its setting.

A way to specify a validation routine could certainly defer back to the original sub, with opportunity to flag it as optimizable to suppress the original call in the simplest case of (non-localized) sassign.