Unifying namespaces of @name and $name to simplify dereferencing?

[LanX]

This is a meditation !

Preliminary thoughts

As a beginner one of my biggest obstacles with Perl was the duality of the list form @arr =(1,2,3) and reference form $arr_ref = [1,2,3] of data structures (same with %hashes )

see also Re: disadvantages of perl

I should choose references far more often, but I'm normally too lazy to always type the dereference operator -> , that's a breach in the paradigm of "making easy things easy"!

Theoretical approach

let's suppose we had a feature called use feature "autoref" (not sure about the best name)

and defining my @name would automatically involve defining my $name = \@name and vice versa (same with %hashes ).

This could simplify notation a lot.

I not only rarely have collisions between those namespaces, I actively avoid them (like using plural for array names).

Questions

Are there any pitfalls I didn't think of regarding this hypothetical feature?

[Tux]

I think that it would make maint a LOT harder (decreased readability). If people already complein about using the same "name" for variables that are related, this would only worsen that. Personally I'd choose an opt-in to make use of this new hypothetical feature a fatal warning :]

my %name;
my @name;

sub name {
    my $name = shift;
    exists $name{$name} and return $name{$name};
    grep { $_ eq $name } @name and return $name;
    return "invalid";
    } # name

FWIW, the above code is fine with me :)

[LanX]

I don't get it.

> would make maint a LOT harder

maint = maintenance ?

>

sub name {
    my $name = shift;
    exists $name{$name} and return $name{$name};

well obviously you are passing a hash ref in this model because of the curlies {}

but then

>

    grep { $_ eq $name } @name and return $name;

this would mean grepping for a circular reference in the array???

In standard Perl would be

>

    grep { $_ eq $name } @$name and return $name;

Not sure if this really makes sense...

update

BTW: you are still free to use -> in this model to improve readability.

NB: -> is already optional between parens... $a->{x}[0] == $a->{x}->[0]

[shmem]

sub name {
    my $name = shift;
    exists $name{$name} and return $name{$name};

well obviously you are passing a hash ref in this model because of the curlies {}

No. The name passed in (update: in normal behavior, obviously) is a string, the $name{...} lookup is into a previously defined hash:

my %name;
my @name;

</nitpick>

Update: it is in your model where confusion arises about the type of the passed $name.

[Eily]

There would be a collision issue if you had both a hash and an array with the same name, but since this happens mostly because of a bad coding style, or as a mistake, I suppose that's only half an issue. And you could always overload the operators to make your scalar act like a reference to any other type.

In the end, your proposition is just a trick to emulate the sigil invariance of perl 6 isn't it? That's the main issue I think. It would make people coming from other languages (php would definitely be an issue) think they understand what's happening when you write my $array = ["Hello", "World"]; print $array[0];. And it would hide away most notions of list from perl because you could work with a subset of the language that only handles scalars, without any indication in the syntax that thing are not what they seem. If people coming to perl from other languages start using that feature because it makes perl "act normal", the @ sigil would mostly be understood as giving the size of the array, with bugs sometimes appearing when people use it in list context without understanding what list context is.

Maybe if the scalar associated with @array was $array_ref, or some other change that makes it clear that $array[0] is not just a shortcut for $array->[0] that would be OK, but then maybe the extra -> isn't too much pain. And even then I think associating variable on an individual basis instead of an automatic one would be better, bind (my $array_ref => my @array); if my @array; my $array_ref = \@array; is too cumbersome for you (I tried, bind my ($array_ref => @array) doesn't work with prototypes).

Edit: turned my nearly one-paragraph-long sentence into multiple ones, mostly by adding dots :)

[LanX]

Hello Eily,

Thanks for meditating! =)

> There would be a collision issue if you had both a hash and an array with the same name,

well already an immediate compile time error, just take the first two lines of Tux suggestion:

use feature 'autoref';
my %name;
my @name;

translated to

my %name; my $name = \%name;
my @name; my $name = \@name; # <- compile time error

and even if my $name = \%name; comes from an upper scope, one could think about emitting a warning after defining my @name in a lower scope.

(But yes we would have overlaps from all namespaces, the title of this thread is too simplistic.)

> In the end, your proposition is just a trick to emulate the sigil invariance of perl 6 isn't it?

not really, it's another way to address the same question, but I'm looking for a way which does less sacrifices to Perl5 mechanisms like context and could be easily implemented.

Perl6 does not only a complete break to backwards compatibility, but has to redesign context and other stuff completely.

> think they understand what's happening when you write my $array = ["Hello", "World"]; print $array[0];

well if $array is an alias of \@array they would not only think to understand what is happening, the knew what is happening, because $array[0] and $array->[0] would do the exactly same thing.

> with bugs sometimes appearing when people use it in list context without understanding what list context is.

I'm not trying to eliminate list context, one would still need to understand what @arr = (@arr1,@arr2) does.

But yes,it would be much easier for beginners before getting there (e.g. when coming from JS)

[Eily]

I'm not trying to eliminate list context

Well yes, that's not something I would suspect of you anyway ;-).

I actually didn't think the my $array = ["Hello", "World"]; print $array[0]; example through. It might work here because of the my, which means that $array isn't already linked to another array. But if you want to redefine the content of the array without breaking the link you have to write @array = (1..4);. Because $array = [1..4]; would link $array to a brand new array, or not be consistent with "normal perl" (and $ar = $br would either alias @ar to @br, or not lead to $ar == $br, or make $ar != \@ar). So in the end you're right, this doesn't make it easy to just pretend the @ sigil doesn't exist, and this doesn't work like the perl6 sigil invariance at all. I suppose it's fine if the option is scoped correctly :

my @outer;
my $otter = [1..4];
{
  use autorefs;
  my $inner = [1..10];
  print $inner[5];
  # Neither $outer nor $otter[0] are legal
} # @inner is obviously not allowed here

I haven't worked much with perl6, just done a few tests and mostly read about it. I'm sure the subject must have been covered many times but your meditation just made me realize the point of keeping the @ sigil when $a = 1..4; $a[0] and @a = 1..4; @a[0]; do the same thing, and what it changed for @a to be an array and $a to be an array ref if both where just going to be passed as a single argument anyway. That's because @a = 1..4; @b = @a is "equivalent" to $a = 1..4; @b = $a (@b is a clone of the array a) but not to $a = 1..4; $b = $a; or even @a = 1..4; $b = @a ($b is a ref to the array a).

[shmem]

Are there any pitfalls I didn't think of regarding this hypothetical feature?

This would allow either the association of $identifier with @identifier or %identifier, but not both. To allow both, hashes and arrays have to collapse into one data structure, which is what php does. I personally detest this particular php feature. Associating both makes sigils meaningless imho and kicks you back into C where an identifier can hold an address or a pointer to anything including a function, making it impossible to grasp at first glance what the identifier is about. But this is one of the very reasons sigils denoting variable types were introduced in perl (borrowing that from basic) - which makes your suggestion going against the camel's hair, IMHO.

It would make a good perl-critic rule to disallow the use of the same identifier for different purposes, though.

[LanX]

Hello Shmem

Thanks for meditating! =)

> This would allow either the association of $identifier with @identifier or %identifier, but not both

yes I was sloppy with the "vice versa" part in my OP.

> To allow both, hashes and arrays have to collapse into one data structure,

No, no, no that is not my intention.

Perl has the advantage that $identifier is an universal untyped scalar which can hold any string, number or reference.

I'm understanding now that for the reverse direction (assigning a ref) we'd need a declaration clarifying the type, this could be

my $identifier[] = [1,2,3]

and/or

my \@indentifier = [1,2,3]

, both¹ with the same effect of aliasing $indentifier to \@indentifier

So using something like

my ( $name, \@name, \%name ) = @_

would cause 2 redeclaration errors at compile-time because $name is already taken, so no need for a unifying data structures like in PHP, it's just an aliasing trick to simplify syntax of common data structures.

Besides this opens the possibility of run-time checking of assigned data types and consequently further OP-code performance optimizations.

¹) not sure about the best syntax, but both possibilities can't conflict with older code because they are illegal ATM.

... Other suggestions?

[oiskuu]

In perl4 you needed sigils on &subs, I think nobody is pining for that feature. I'd welcome the trend to continue to lexicals as well (let's call it perl7 ;-). Lexicals are good. More scoping means better structured code, opportunities for high-level optimizations. Simple variables need no symbolic bindings: compiler could optimize their allocation, etc.

Already there are experimental features: my $foo = sub vs my sub foo. That doesn't quite complete the circle since there's no implicit dereference ($foo() as $foo->()). But what if lexical subs were just sigil-less constant references? Naturally, subs in global scope ought to default to lexicals, unless marked as our.

References tend to encourage good practices (structured data, passing refs vs lists in/out of subs), and they can be easier to understand (array flattening). As it stands, mixing arrays and lists come with many nasty pitfalls. E.g. differences in slicing. E.g. an array-returning routine has a special case:

sub foo {
#   return @_ ? @_ : @{[7, 76543]};
   return @_ ? @_ : (7, 76543);
}

say "as a list: ", join " ", foo(1, 2, 3);
say "as int: ", int foo(1, 2, 3);

say "as a list: ", join " ", foo();
say "as int: ", int foo();

I've been thinking the bare lexicals could be reference-anything variables, whereas sigils would place constraints. Sigils might also serve as prototypes in sub declaration. Ah yes, but this is starting to look like reinventing perl...

my foo = either scalar or reference;

foo[] foo() foo{} foo[](){}[];     # implicit -> and autovivify

Where subs autovivify means autoloading or weak binding (to core library).

Anyway. Thank you for your thoughts.

[LanX]

BTW: there is a subtle difference between @arr=(1,2,3) and $arr=[1,2,3] , the first is reusing the allocated space (eventually expanding) to store the list , the latter assigning a ref to a newly allocated memory location.

One needs both!

In a language without sigils you have to invent list mechanisms to implement this.

[LanX]

Hello Oiskuu,

Thanks for meditating! =)

I like the idea of sigil less variables, but I doubt this is feasible in Perl, cause there are too many cases where context matters, so you can't avoid @ and % to denote it...

... like in %settings = (%defaults, %arguments)

Perl without sigils would be a kind of JavaScript (which isn't bad, just very reduced)

I always thought of sigils as a way to spare me of Hungarian notation, in order to understand which type a variable has.

Also sigils help avoiding namespace collisions with builtins, like in $length .

> As it stands, mixing arrays and lists come with many nasty pitfalls.

That's why I was meditating about a very simple change, to implicitly replace any my @arr; with my @arr;my $arr = \@arr; .

This could be implemented easily (e.g. macro mechanism) and the effects are obvious.

Though I was naive with the vice versa part, because my $var could be any datatype, so aliasing my @var to @$var wouldn't be obvious.

And one of the most important use cases is passing data structures around, so my $arr = shift would cause a dangerous ambiguity when forcing an automatic alias.

Though an extended syntax like sub foo { my $arr[] = shift; ... } (or my \@arr = shift ) might solve this and implicitly alias @arr (or $arr respectively).

UPDATE

I'm just realizing that something like my $arr[] = shift has an extra benefit of allowing runtime errors if the wrong type is passed to $arr, while writing my $arr = shift would allow to pass whatever you want. :)

PS: As a side note:

I was once meditating of adding new sigils for €arrrefs and £hashrefs but this is not only problematic with character encodings but also with keyboard designs.

Just think of The Damians suggestion of appending _ref to each $name. What I do is to prepend $a_... , $h_... , $c_...' . This could be avoided if we had sigils for those references.

[oiskuu]

Well, I did not mean to suggest we do away with sigils altogether. These are just too convenient, if only for string interpolation.

To illustrate the possibilities: let's say we have the statement

my foo = [ 1, 2, 3 ];

This shadows all other symbols foo in scope. Further, any $ @ % sigils on foo are just shorthand for ${} @{} %{}. Now you can write

print @foo;

... and have it work DWIM. Or use the bare foo as reference. But the problem is, if foo refers to code you'll need explicit () to make the call. Hmm. And then, is $doh->foo() a method call or autoboxing? There are probably many caveats I can't think of. So you may be right that this feature wouldn't integrate well.

Anyhow, the more compact syntax would see more use. Isn't this all about hubrislaziness after all?

2017-04-01 Embarrassing. I think laziness but write hubris, some times. Blame perl!

[Tux]

Did you see the refaliasing feature? Not that I'd ever use it, but it seems to address your wishes.

Read the the full details, available since 5.22.0

[LanX]

Oh thanks that's indeed interesting, Perl core finally allowing aliasing of lexicals! :)

... not sure if it solves the all issues yet...

But you know these new possibilities deserve a proper thread ...

update

hmm ... coupling is broken after assigning a new ref

use 5.22.0;
use feature qw/say refaliasing/;
no warnings "experimental::refaliasing";

\my @a = my $a = [666];
say "noref: $a[0], ref: $a->[0], list: @a";    
$a = [42];
say "noref: $a[0], ref: $a->[0], list: @a";

-->

noref: 666, ref: 666, list: 666
noref: 666, ref: 42, list: 666

update

at least passing arrays to arrays is possible now

use 5.22.0;
use feature qw/say refaliasing/;
no warnings "experimental::refaliasing";

my @a = (666,42);

test (\@a,\@a);

sub test {
    (\my @sub, my $ref) = @_;
    say "noref: $sub[0], ref: $ref->[0], list: @sub";
}

-->

noref: 666, ref: 666, list: 666 42

well the syntax when passing more than one argument is not very elegant, see duplicated my ... but at least it's consistent.

[RonW]

I'm normally too lazy to always type the dereference operator ->

Sounds to me like you are really asking for auto-dereference, so that $a[1] is interpreted as $a->[1]

While auto-magically defining my $name = \@name would simulate something like that, I think it just introduces too many other problems.

Even assuming that the same symbol name is not reused for other types of variables (a good practice in general), is implicit dereferencing a scalar variable really a safe thing to do? If not, then this feature would be setting up Perl beginners for problems when implicit dereferencing needs to be disabled.

[LanX]

> While auto-magically defining my $name = \@name would simulate

It's not only simulating, one has to enforce the control over both namespaces to make it safe and detect conflicts.

> is implicit dereferencing a scalar variable really a safe thing to do?

the implicit my for the bound variable guarantees this.

Many languages do implicit dereferencing, they just can't afford separated namespaces then.

> If not, then this feature would be setting up Perl beginners for problems when implicit dereferencing needs to be disabled.

most of this thread is dealing with the question of how to safely blend with "classical" variables. (which wasn't my initial intention)

My current approach is that a array reference $arr_ref with implicit dereferencing shouldn't allow equally named variables of other types, i.e. if $name[.] := $name->[.] then trying to use $name{} or $name->{} or $name->() in the same scope should fail at compile time, if those variables live in an upper scope they should be hidden in the current scope.

I have problems to imagine code where the same identifier is used for different types, which shouldn't be better refactored.

[RonW]

I have problems to imagine code where the same identifier is used for different types, which shouldn't be better refactored.

Yes, this is a problem.

Unlikely enforcing this would ever become a module, let alone an optional feature of Perl itself. And I don't know if Perl Critic has a rule for this.

[RonW]

Have you looked at Perl6::Variables ? It appears to provide the implicit dereferencing you want - at least when accessing elements of an array/hash. (Along with other Perl6 variable related syntax.)

By itself, it does not enforce single type use of symbol names, but that could be done with another module.

Update: Looking at it more, it doesn't attempt to translate @array to \@array so passing an array ref or assigning to a scalar still requires using \@array - but, once passed or assigned, $second = $aref[1] will work.

However, even with a means to enforce single type use of symbol names, I still question whether implicit dereferencing in Perl5 would always be safe.

Note: Perl6 has explicit dereferencing as well, using . instead of ->