Ok, this is a fair concern. The question is what sort of syntax would work when, say, passing in an argument to a function. One could have implicit assignments in a unitary left hand operator for example, something like:
my $foo = somefunc(\= $bar);
This would be contextually equivalent to %$bar or @$bar depending on what $bar references. The idea would be it means "what ___ references" and that gets coerced into a list or array after that point. Of course in this context, it would be coerced into a list (of possibly only one value)...
In short if the \= operator was overloaded so that it could be either binary or unitary (left-hand), then the problem goes away.