Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

If you believe in Lists in Scalar Context, Clap your Hands

by oshalla (Deacon)
on Oct 23, 2008 at 17:44 UTC ( #719099=perlmeditation: print w/ replies, xml ) Need Help??

There was a learned discussion around this recently 715269 -- from which one concluded... well I wasn't sure...

The humble programmer needs a mental model for what goes on, and some way to describe it. So I wondered, what's wrong with the notion of a List in Scalar Context ?

The business of Context is peculiar to Perl, it's easy to miss first, second and possibly third time through. Bearing that in mind, this is what perldata says:

In a context not requiring a list value, the value of what appears to be a list literal is simply the value of the final element, as with the C comma operator. For example,
@foo = ('cc', '-E', $bar);
assigns the entire list value to array @foo, but
$foo = ('cc', '-E', $bar);
assigns the value of variable $bar to the scalar variable $foo. Note that the value of an actual array in scalar context is the length of the array; the following assigns the value 3 to $foo:
$foo = @foo; # $foo gets 3
So, blink and you've missed a number of things: ...first, the implication that a list (or at least a list value, or possibly a list literal) is not the same as an "actual array"; ...second, we can talk about an "array in scalar context" (so a "list in scalar context" seems natural); and...third, the author is in knots describing something that looks like a list, but isn't a list (and I want to meet the person who understood the fineness of that distinction the first time they read it).

But the notion of a list goes well beyond list literals, and we need a way to think about how lists are handled more generally.

You might think that lists are something to do with '()' as list constructor and ',' as list item separator. (Anonymous lists have the '[]' constructor...) Of course you would be wrong. The '()' are incidental, required because ',' is a sort of operator with lower precedence than '=', in particular. The '()' are not required at all in some contexts, notably following a list operator (eg print !).

Anyway, in the following, the stuff to the right of the '=' appear to be lists:

@r = () ; @r = undef ; # sets $r[0] = undef @r = 1234 ; @r = (12, 23, 34, 45) ; @r = 'a'..'z' ; @s = ('a'..'z')[4..15] ; @t = @r[7..16]; @q = 9..13 ; @u = @r[@q] ; @r{'a'..'z'} = (1..26) ; @w = @r{'k'..'p'} ; @p = 'k'..'p' ; @w = @r{@p} ;
so what happens if we change this to assignment to a scalar, so we have Scalar Context:
$r = () ; # Case 1 -- $r -> undef $r = 1234 ; # Case 2 -- $r -> 1234 $r = (12, 23, 34, 45) ; # Case 3 -- $r -> 45 $r = 'a'..'z' ; # Case 4 -- error $s = ('a'..'z')[4..15] ; # Case 5 -- $r -> 'p' $t = @r[7..16] ; # Case 6 -- $r -> 'q' $u = $r[@q] ; # Case 7 -- $r -> 'n' $w = @r{'k'..'p'} ; # Case 8 -- $w -> 16 $w = @r{@p} ; # Case 9 -- $w -> 16
Case 3 we are asked to understand in terms of the Scalar Context behaviour of the ',' operator, but can equally well be understood as a List in Scalar Context. Case 4 can only be understood in terms of the Scalar Context behaviour of '..' (in this case to whine about the arguments). However, all the other cases are consistent with the notion that there is a list on the rhs, and that in Scalar Context the result is the last item of the list -- and not a ',' operator in sight ! The List in a Scalar Context is holding up, so far...

Of course, we entirely understand that an Array or a Hash in Scalar Context is quite different:

$r = @r ; # Case 1 -- $r -> 26 $v = %r ; # Case 2 -- $r -> 19/32 $r = (@r) ; # Case 3 -- $r -> 26 $v = (%r) ; # Case 4 -- $r -> 19/32
The List in a Scalar Context notion might suggest that case 3 would set $r to the last entry in @r and case 4 would set $v to some value in %r, if you thought that '()' was a list constructor -- but it isn't, so it doesn't. So far so good, but what about:
$r = (@r, @r) ; $v = (%r, %r) ;
If we think about the rhs as a list, we may be disappointed -- in List Context, the @r or %r would be interpolated. That's not what happens here: the @r and %r are evaluated in Scalar Context, first; the result is then consistent with either List in a Scalar Context or Comma as a Scalar Operator views. (Remembering that ',' is just a list item separator, it's not a list constructor, and it has low precedence anyway.)

OK. The mental model so far, in expressions:

  • Context is important (!)
  • in List Context there are a number of things which make lists, including:
    • the empty list
    • singleton values (including undef)
    • literal lists -- two or more values separated by ',', grouped together by '()' if required
    • the '..' range operator
    • slicing other lists, arrays or hashes
  • in Scalar Context, the above constructs:
    • yield the last item of the list (undef for the empty list)
    • except for '..' which does something completely different in Scalar Context
    • noting, carefully that items in the list are evaluated in Scalar Context, if they are evaluated. (This is Perl Magic...)
  • Arrays are not the same as Lists
    • in List Context an array will evaluate to a list.
    • in Scalar Context an array will evaluate to the number of items in the list (more or less).

Now, subroutines: we have to understand that Context has a long arm -- the Context in which a subroutine is called, will, at a distance, apply to any expression yielding a return value. And different calls may provide different Contexts. This is slippery stuff, but at least the results are entirely consistent. If you take the examples above and replace each rhs by a call to a subroutine which returns that rhs, then the results are identical. But, it is worth noting that, for example, replacing:

return ($a, $b, $c, $d) ;
by:
my @return = () ; ... push @return, $a, $b ; ... push @return, $c, $d ; ... return @return ;
changes the Scalar Context result in a way that may slip your mind as you make an apparently straightforward change !

This is telling us something. A subroutine that is serious about being called in either Scalar or List Context, must ensure that it returns an explicit, well defined result in both cases -- which for Scalar may be the same as Array in a Scalar Context, or List in a Scalar Context, or anything else it cares to return. Further, it is foolish to call a subroutine in a Context which it does not explicitly support -- which implies that it is foolish to call a subroutine defined only to return a list, in a Scalar Context, and expect List in a Scalar Context semantics.

This is also true of Perl functions. There is no point assuming that a Perl function that returns a list in List Context will do anything useful at all in Scalar Context, except where it is defined to do so.

So, after all this (and thank you for reading this far!) am I clapping my hands ?

Well, yes I am. It seems to me that List in a Scalar Context is a perfectly good way of understanding the behaviour of lists in expressions, as discussed above.

However, the application is limited. In particular, it cannot be used to predict what a list returning operation will do if you decide to use it in Scalar Context. To that extent, the problem with the notion of List in a Scalar Context is that it is deceptively general, and dangerously so.

My conclusion: List in Scalar Context is a more general notion than Comma in Scalar Context, but you have to understand its limitations. And, it is much more important to understand that knowing that a Perl function or a subroutine returns a List in List Context, tells you nothing useful about its Scalar Context behavious.

I shall continue to clap, but in a quiet and restrained manner, as befits someone tip-toing through the minefield that is Perl Semantics !

BTW: I'm gagging to know how to describe why this:

$r = () = 1..27 ;
does what it does.

Finally, and for extra points (and points mean prizes), how does one describe the difference between a list and an array ?

Comment on If you believe in Lists in Scalar Context, Clap your Hands
Select or Download Code
Re: If you believe in Lists in Scalar Context, Clap your Hands
by moritz (Cardinal) on Oct 23, 2008 at 18:57 UTC
    Finally, and for extra points (and points mean prizes), how does one describe the difference between a list and an array ?

    That's not that hard. Anything that you can push onto is an array, all other things that look like arrays or lists are actually lists. (Any array method that changes the length works as a probe instead)

    Examples for Arrays:

    @array # this works: push @array, $item;

    Examples for Lists:

    # things returned from subs: localtime(); # slices: @INC[0,1,-1]; # literals qw(foo bar); # return values from operators ('12') x 10; # you can't push() on any of these - and it wouldn't make sense anyway

      OK. Lists cannot be lvalues, though you may have a list of lvalues...

      Suppose I refine the points paying question: as an rvalue how does one describe the difference between a list and an array ?

        An array is everything that could be used as an lvalue (perhaps with the exception of :lvalue subs, haven't tried those), independently of you actually use it as an lvalue.

        That definition so far helped in all real-world cases that I've look at.

        Lists cannot be lvalues, though you may have a list of lvalues...

        Say again? An lvalue list returns a list of lvalues and enforces list context, just like an rvalue list returns a list of rvalues in list context.

        In both of the following snippets, you'll see an list as an lvalue ("M" flag).

        >perl -MO=Concise -e"($a,$b)=(4,5)" a <@> leave[1 ref] vKP/REFC ->(end) 1 <0> enter ->2 2 <;> nextstate(main 1 -e:1) v ->3 9 <2> aassign[t3] vKS ->a - <1> ex-list lKP ->6 <-- RHS list 3 <0> pushmark s ->4 4 <$> const[IV 4] s ->5 5 <$> const[IV 5] s ->6 - <1> ex-list lKPRM* ->9 <-- LHS list w/ "M" 6 <0> pushmark sRM* ->7 - <1> ex-rv2sv sKRM*/1 ->8 7 <#> gvsv[*a] s ->8 - <1> ex-rv2sv sKRM*/1 ->- 8 <#> gvsv[*b] s ->9 -e syntax OK
        >perl -MO=Concise -e"1 for ($a,$b)" d <@> leave[1 ref] vKP/REFC ->(end) 1 <0> enter ->2 2 <;> nextstate(main 1 -e:1) v ->3 3 <;> nextstate(main 1 -e:1) v ->4 c <2> leaveloop vK/2 ->d 8 <{> enteriter(next->9 last->c redo->9) lK/8 ->a - <0> ex-pushmark s ->4 - <1> ex-list lKPM ->7 <-- lvalue list 4 <0> pushmark sM ->5 - <1> ex-rv2sv sKM/1 ->6 5 <#> gvsv[*a] s ->6 - <1> ex-rv2sv sKM/1 ->- 6 <#> gvsv[*b] s ->7 7 <#> gv[*_] s ->8 - <1> null vK/1 ->c b <|> and(other->9) vK/1 ->c a <0> iter s ->b - <@> lineseq vK ->- - <0> ex-const v ->9 9 <0> unstack v ->a -e syntax OK
Re: If you believe in Lists in Scalar Context, Clap your Hands
by BrowserUk (Pope) on Oct 23, 2008 at 19:32 UTC

    My (latest :) final word on this is:

    Describe the difference between the following two lines of Perl:

    • print 'this is a label: ' , @a;
    • print 'this is a label: ' . @a;

    Can anyone, hand on heart, say that their first--and in my case, abiding--thoughts don't contain the phrase: array in a scalar context?

    If you are aware enough of Perl, to be aware of context, that's what you are going to think. And it doesn't really matter what political, ideological, or technical restraints might prevent you from saying that aloud, or what convoluted alternative description you may construct, that's going to be your first thought.

    And if you substitute: ('a','b','c') for @a, then you just switch from 'array' to 'list' in the mental image. It holds. It works. It's simple and intuitive.

    And regardless of whether you feel you have been seduced by any technically inaccurate minutiae that might be inherent in the phrase: list in a scalar context, all the alternatives (and actually very few, if any have been offered), suck! Bigtime!


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re: If you believe in Lists in Scalar Context, Clap your Hands
by JavaFan (Canon) on Oct 23, 2008 at 19:58 UTC
    I've -- your posts. You seem to advocate the "lists in scalar context" meme. Context in Perl is important, and while confusing for some beginners, not that hard. Except when you're lured into thinking "lists in scalar context".

    The fact you're troubled by context (and related things) is proven by your final question:

    Finally, and for extra points (and points mean prizes), how does one describe the difference between a list and an array ?
    This is actually quite easy. "Arrays" are something the Perl runtime really knows about. They are represented as AVs internally. They can be sliced and diced, passed around, changed, etc, etc. "Lists" however are a human thing - to make it easier to talk about code. Perl doesn't know about lists. It knows about list context. But what humans call a "list" is for Perl just a sequence of scalars in list context.

    And once you understand that, you never think about lists in scalar context again. It just doesn't make sense. Lists are there because the context is "list context".

      I've -- your posts.

      You felt the need to downvote them all?

      And I've downvoted your post for .... (I'll fill that in with an PC epithet later maybe.)

      Perl doesn't know about lists.

      What utter clap trap! perl may not know about lists, but Perl surely does.

      Else, how do you explain all the references to lists in the perl documentation? 244 in perlfunc alone. And thousands in the document set as a whole.

      And human beings use Perl. What goes on inside perl isn't, (and should not be) of any interest to the vast majority of Perl users. Weirdos like we two aside.

      All your talk about "AVs" and and "the internals" are far more damaging to Perl and novice Perl users, than any mild technical inaccuracy in a conceptual model that fits well with both the documentation, ethos and apparent behaviour of Perl.

      Perhaps that's why you call yourself a java fan?


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
      Oh really?
      my @stuff = ("this", "that", "and", "the", "other", "thing")[0, 1, 5];
      It looks like I just sliced a list.
      print "Hello " . ("earthlings", "world");
      It looks like I just put a list in scalar context.

      However I agree that you can't do the other things you said you can't do without constructing something other than a list. Furthermore I'm the kind of person who never thought much of semantic arguments. I don't care what you call the construct, as long as you understand what it does.

        Quick! Bandaid that bleeding list! :p

      OK. The implementation detail is obviously interesting, and one would hope that it is strongly related to the concepts it embodies.

      Taking a listy kind of Perl function more or less at random:

      • map BLOCK LIST
      • map EXPR,LIST
      Evaluates the BLOCK or EXPR for each element of LIST (locally setting $_ to each element) and returns the list value composed of the results of each such evaluation. In scalar context, returns the total number of elements so generated. Evaluates BLOCK or EXPR in list context, so each element of LIST may produce zero, one, or more elements in the returned value.

      I'm curious how this is to be understood if Perl doesn't know from lists.

        I didn't know Perl read the manual page.

        As I said, "List" is a concept for humans. The fact that LIST is mentioned in the manual page only supports that notion - after all, it's humans who read the manual page.

        For Perl, map just has a sequence of scalars as arguments.

        What I think JavaFan is getting at when he says "lists in scalar context" is a damaging meme is that there is no such thing, and he feels that this conception leads to serious errors of understanding.

        See perlop for why there is no such thing:

        A named array in scalar context is quite different from what would at first glance appear to be a list in scalar context. You can't get a list like (1,2,3) into being in scalar context, because the compiler knows the context at compile time. It would generate the scalar comma operator there, not the list construction version of the comma. That means it was never a list to start with.

        There's a bit more in the 'Comma Operator' section of perlop.

        So, when you have
            $r = (1, 5, 10)
        perl sees something like
            $r = 1 return_right_side 5 return_right_side 10;
        and
            @r = (1, 5, 10)
        becomes something like
            @r = list_constructor( 1 list_separator 5 list_separator 10 );


        TGI says moo

Re: If you believe in Lists in Scalar Context, Clap your Hands
by chromatic (Archbishop) on Oct 23, 2008 at 22:12 UTC
    What's wrong with the notion of a List in Scalar Context ?

    There are many ways to produce a list in Perl, but there is no general rule for the result of evaluating a list in scalar context. There is a general rule for the result of evaluating an array in scalar context, which is why it's important to distinguish between lists and arrays.

      but there is no general rule for the result of evaluating a list in scalar context.

      The general rule is that the last item of the list is returned, but there might be exceptions (as always ...).

      The OP mentions that $scalar = 'a' .. 'z' is an error, but that's really a feature of the range operator, not of "list is scalar context".

      The only exception I'm aware of is $r = () = 4..5, where the number of items is returned rather than the the last item. But by my definition above that would actually be an array, not a list, because you can assign to it.

      Are there cases where something that's unambiguously a list (and not an array) returns anything else than the last item?

      (And I'm not talking about functions that are context sensitive via wantarray either).

        The only exception I'm aware of is $r = () = 4..5, where the number of items is returned rather than the the last item. But by my definition above that would actually be an array, not a list, because you can assign to it.
        But it ain't an array - you're assigning to a(n empty) sequence of scalars. And that $r ends up with the number of items has nothing to do with lists in scalar context. It's a feature of the assignment operator. Since () is an empty list, the second '=' is the list assignment operator. And the list assignment operator in scalar context returns the number of elements on its right hand side. (See "man perlop")

        On $r = () = 4..5; vs @r = () = 4..5;

        It's useful to look at this snippet to help understand what's happening:

        $r = ($x, $y) = (1..10);
        • Assignment is right associative.
        • ($x,$y) = 1..10 sets $x, and $y and returns in scalar context returns the number of elements on the right hand side (in this case, 10). In a list context it will return a list of lvalues assigned to.
        • So now we have reduced the above to $r = 10 which needs no explanation.
        • Note that it doesn't matter what is in the middle term, as long as it puts the right hand assignment in list context.

        If we look at @r = () = 1..10 we will see that the right hand assignment assigns to no lvalues and so returns an empty list. And thus @r is empty.


        TGI says moo

        Are there cases where something that's unambiguously a list (and not an array) returns anything else than the last item?

        A regular expression match with the /g flag, in scalar context.

        A map or grep expression in scalar context.

        The empty list my $x = () = some_expression; construct, in scalar context.

        split, chomp, ....

        A bare hash. (Want to argue over what a list is? Is it multiple, comma-separated expressions or something that pushes one or more items onto the internal stack? I'm not sure lists even exist as a language-level construct in Perl 5.)

Re: If you believe in Lists in Scalar Context, Clap your Hands
by ikegami (Pope) on Oct 23, 2008 at 22:55 UTC

    First, some corrections:

    $r = 'a'..'z' ; # Case 4 -- error

    Maybe, but $r = 5..7; wouldn't be.

    $r = (@r, @r) ; $v = (%r, %r) ;

    the @r and %r are evaluated in Scalar Context

    No, they're not.

    use strict; use warnings; sub print_context { my $c = ( wantarray() ? 'List' : ( defined(wantarray()) ? 'Scalar' : 'Void' ) ); print("$c\n"); } my $r1 = ( print_context(), print_context() ); # Void # Scalar

    However, they would be evaluated in scalar context in the following similar code:

    my $r2 = sub { ( print_context(), print_context() ) }->(); # Scalar # Scalar

    Now, some questions for you:

    in List Context there are a number of things which make lists, including:

    • the empty list
    • singleton values (including undef)
    • literal lists -- two or more values separated by ',', grouped together by '()' if required
    • the '..' range operator
    • slicing other lists, arrays or hashes

    How come array slices and hash slices are in that list, but not arrays and hashes?

    Why aren't any named operators (such as grep and map) in that list?

    what's wrong with the notion of a List in Scalar Context ?

    Given that an array returns a list in list context, how do you place that list in scalar context?


    Finally, an answer to your question:

    BTW: I'm gagging to know how to describe why this:

    $r = () = 1..27 ;

    A list assignment ("() = 1..27") in scalar context ("$r =") evaluates to the number of elements to assign.

    Finally, and for extra points (and points mean prizes), how does one describe the difference between a list and an array ?

    An array is variable.
    A list is a piece of code that generates a list operator.
    A list is the operator generated from a list in the code.
    A list is the result of an operator or function in list context.

      The difference you point out between:

      my $r1 = ( print_context(), print_context() ); # Void # Scalar my $r2 = sub { ( print_context(), print_context() ) }->(); # Scalar # Scalar
      is amusing -- not quite as consistent as I thought.

      Now, some questions for you:
      in List Context there are a number of things which make lists, including:
      ....

      How come array slices and hash slices are in that list, but not arrays and hashes?

      Why aren't any named operators (such as grep and map) in that list?

      OK. The list was things that make lists, which are the problematic things Scalar/List context-wise. At that point I was summarising what had been discussed so far... I went on to subroutines and named operators. Sorry if that was less than clear.

      what's wrong with the notion of a List in Scalar Context ?
      Given that an array returns a list in list context, how do you place that list in scalar context?

      I would argue that, when treated as an rvalue, this is the key difference between a list and an array. Since in Scalar Context the value of the array isn't the list, the case does not arise.

      I understand that it can also be argued that in Scalar Context there are no lists... however much it looks as though there might be.

      That the value of @array depends on the Context is one thing to understand. That ($a, $b, $c) or @z[@p] are or are not lists, depending on the Context, seems to me to be harder to get to grips with, and unnecessary.

      Finally, an answer to your question:
      BTW: I'm gagging to know how to describe why this:
      $r = () = 1..27 ;
      A list assignment ("() = 1..27") in scalar context ("$r =") evaluates to the number of elements to assign.

      Ah. I had expected the second, scalar assignment, to operate on the result of the first assigment. I have now found the place in the book where, by definition, a scalar assignment to the left of a list assignment gets the number of items to the right of the list assignment, not the number of items successfully assigned in that list assignment. (Well, that's clear to me now !)

      Thank you. Perl, eh :-)

      Finally, and for extra points (and points mean prizes), how does one describe the difference between a list and an array ?
      An array is variable. A list is a piece of code that generates a list operator.
      A list is the operator generated from a list in the code.
      A list is the result of an operator or function in list context.

      OK. Well, everybody avoided mentioning the influence of Scalar Context on arrays. But then if lists can only exist in List Context, that wouldn't be worth mentioning :-)

        The list was things that make lists, which are the problematic things Scalar/List context-wise. At that point I was summarising what had been discussed so far...

        But you had discussed arrays and hashes. Are you implying that array slice make lists and arrays don't?

        The omission seems rather "convenient" because it allowed you to cut two exceptions from the "in Scalar Context" list.

        Given that an array returns a list in list context, how do you place that list in scalar context?

        I would argue that, when treated as an rvalue, this is the key difference between a list and an array. Since in Scalar Context the value of the array isn't the list, the case does not arise.

        Exactly. A list can't be produced in scalar context. So how can a list be put into scalar context?

        The point of the question was to point out that "list" is an overloaded term. "List in scalar context" is deceptive, misleading and/or confusing.

        Ah. I had expected the second, scalar assignment, to operate on the result of the first assigment.

        Assignment operators are right-associative. That means that in chain of assignments, the right-most has highest precedence. ** is also right-associative.

        >perl -le"print( 2**3**4 )" 2.41785163922926e+024 >perl -le"print( (2**3)**4 )" 4096 >perl -le"print( 2**(3**4) )" 2.41785163922926e+024

        OK. Well, everybody avoided mentioning the influence of Scalar Context on arrays.

        Two of the three types of list I mentioned in my answer can't be put into scalar context. That makes it a rather minor difference.

Re: If you believe in Lists in Scalar Context, Clap your Hands
by TimToady (Parson) on Oct 23, 2008 at 22:56 UTC
    What a confusing language! We really should fix it some millennium or other--preferably this one. So if you happen to invent a time machine (or even if one just appears on your doorstep from the future), please let me know, because some of us are planning to go back to the year 2000 and perform a brain transplant on Larry so he starts thinking straighter. :)
      perform a brain transplant on Larry

      Bad idea. Very bad idea. The transplant might not well resonate with what is Larry, and Larry might start inventing really weird stuff: not only unreasonable, but unresonating, i.e. cacophonic as well. *shudder*

      Keep that one, Larry, at all times. Please. We love you nonwithstanding, and the language is fine that way.

        Keep that one, Larry, at all times. Please. We love you nonwithstanding, and the language is fine that way.

        I personally believe I'll second that, sh-meme, all the way, all the time: incidentally, in Italian we use the word "tempo" both for "time" and "weather" so speaking of resonating thingies, I for one would (half-heartedly) prefer the efforts gone into the creation of a time machine to be spent for the creation of a weather one, that would give us a wonderful sunshine as the basis for a reggae-like positive vibration which in turn would allow us to dance all time with a glaring smile on our faces! But if a time machine were actually to exist too, I admit it would be exciting: however I stress the "too" because it would not be bad to have it along with that other machine, and then I'd go back in some historic period in which men were so busy hurting each other and I'd try to give them the sunshine, and the positive vibe, and... Perl of course! And seriously... well, sort of seriously: the best year to go back to would be 1956 rather than 2000. And once there, the language which would be to seriously undergo a redesign is FORTRAN which is fine per se: but taking into account its heritage, just think how present would be if it were more perlish to start with!

        If you think that the above is pure madness... well, then you're right. However one thing to notice is that while there do exist programming languages (and I thought I had "collected" more, but now I can't find them offhand on my delicious) aimed at real time artistic creation, which I think is the reason why all this is in topic here... and even I'm sure to have stumbled upon a Python extension in that realm too... nothing similar seems to exist for Perl, which saddens me since I believe its higher expressive power would be incredibly well suited for such tasks too!

        --
        If you can't understand the incipit, then please check the IPB Campaign.
Re: If you believe in Lists in Scalar Context, Clap your Hands
by tinita (Parson) on Oct 24, 2008 at 11:20 UTC
    I think the problem is program code vs. execution.

    Look at this code:

    sub foo { ... return ($x, $y, $z); }

    By the definition, this sub returns a list. We all know that during runtime, it might not be a list, but how else would you describe what the sub is returning shortly?

    Now this code:
    my $x = foo();
    This is a function call in scalar context. Now, together with the definition of the subroutine above, the sub somehow returns the list in scalar context. By definition. Which, during runtime turns out to be a scalar, because the list never existed, because perl knows the context early enough. So I would agree there is no list in scalar context during runtime, but for me it's clearly describing what code does. Whenever I read another post "There is no list in scalar context" I think, *sigh*, yeah, there might never exist such a list, but it's a good description of the behaviour.

      By the definition, this sub returns a list.
      No, it returns the value of the last expression, evaluated in the context the function was called in. In your example, the expression is the comma operator.

      Note also that:

      sub look_ma_I_am_returning_an_empty_list { return (); }
      actually returns a value in scalar context: the undefined value.
        it returns the value of the last expression, evaluated in the context the function was called in

        For a start, you'd have to modify that with s[was][will be].

        Like I say, the alternatives suck! Bigtime!


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
        No, it returns the value of the last expression, evaluated in the context the function was called in.
        I know that, you know that, most of the others know that. Where's the problem?For me, a list(!) of scalars with the comma operator in between is a list. In english. To say that my example sub returns a list (I could even say here *every* sub returns a list) isn't saying anything wrong. If there is no list at all in perl, then it's clear I mean the english wort list. In my example I could also say three scalars.

        The point I want to make is whenever someone explains a piece of code with "list in scalar context" then you can be sure somebody who has too much time will correct them. But they didn't have the time to explain the piece of code in the first place. When you're talking about code, do you always use that long definition with the comma operator? If there is no list in perl, do you really *never* use this word when talking about code? Come on. I want to know what's wrong with using that word when explaining others the behavior of a piece of code. Didn't you get my point about code vs. execution?

        edit: or maybe your suggestion is, never use the word list, but use comma operator. I'm not sure if it's the right thing to always say "sub x returns the comma operator" when a) a list is a word that everybody understands and b) a list is used as a synonym for comma operator, so it doesn't say anything wrong. Couldn't we just say that? I never see where the explanation "list in scalar context" is saying anything that is not true.

Re: If you believe in Lists in Scalar Context, Clap your Hands
by merlyn (Sage) on Oct 24, 2008 at 21:59 UTC
    The problem here I think is that people are confusing two concepts. The syntax of values separated by commas is called a list. But so is the value in memory consisting of many other scaler values. When I say you cannot have a list in a scalar context, I mean a list value. However, when most people say list and think of the things separated by commas, of course that is also a list to them. However, at no point is that list ever actually a list value. So before you argue this to death, please be clear in your terminology. The representation of the list as values separated by commas could indeed be in a scaler context, but this would never ever be an actual list value at any time during the execution. With that in mind, the answer is that no there never is a list of value in a scalar context: it is an impossibility.
Re: If you believe in Lists in Scalar Context, Clap your Hands
by ig (Vicar) on Oct 26, 2008 at 22:28 UTC

    I was clapping until I stopped to wonder what a list is. Now I'm confused.

    I can't find a definition of what a list is. The term is much used but the lack of definition is troubling. More so when considering phrases including "list" such as "list value" and "list context". Much can be inferred from what is said in the documents, but that isn't the same as having an authoritative definition. Am I missing one somewhere? I expected one in perlsyn and found none. I found most in perldata but still not a clear definition.

    It appears, from various statements about or refering to lists in perldata that there is no syntactic construct that is necessarily a "list", even the syntax is dependent on the context, otherwise why brain twisters like:

    In a context not requiring a list value, the value of what appears to be a list literal is simply the value of the final element, as with the C comma operator.

    And I note in passing that this suggests not only that what looks like a list may not be, depending on the context, but also that what looks like the comma operator in something that would be a list in a context that required a list value but isn't because the context doesn't require it is not the comma operator but merely something that causes evaluation "as with" it.

    So, now I know what I thought a list was and, for the sake of simplicity and sanity, what I wish it reliably and consistently was (i.e. a syntactic construct that is a syntactic list regardless of context with semantics that depends on context - what most recognize as "looks like a list"), but I don't know what a list is in Perl and seem to know less the more I read and think about it.

    Maybe it is true that lists only exist in contexts requiring list values. If this is true, what is that thing that looks like a list but isn't because of the context?

    Acceptance of the phrase "looks like a list" suggests there is a commonly understood underlying concept of list that Perl deviates from. It is unfortunate that there is a deviation from this commonly understood concept. It is worse that the deviant concept is so inimical to definition that no positive description is offered nor even a label other than the phrase "what appears to be a list". This deviant concept is, I think, at the root of the controversy, and as long as there is no definition of it the controversy will remain.

    But I have enjoyed reading all the opinions and examples, and learned to be more careful in my thinking, so thanks all...

Re: If you believe in Lists in Scalar Context, Clap your Hands
by ysth (Canon) on Oct 27, 2008 at 16:17 UTC
Re: If you believe in Lists in Scalar Context, Clap your Hands
by ysth (Canon) on Oct 28, 2008 at 04:36 UTC
    So, what value do you see in advocating this model?

    I prefer to stick to models that, well, model what actually happens. And what actually happens is that the , operator really does have a scalar-context mode that gives the left operand void context and the right operand scalar context.

    However, in the case of a literal list returned by a sub, there are complications. Context in perl is somewhat limited. It can either be derived at compile time or looked up at runtime. If the latter, it is the context in which the current sub, eval, or mainline code is called (which itself may or may not be known at compile time).

    But this isn't complex enough to fully handle a literal list. When a literal list appears in list context, each component should have list context. This works fine. When a literal list appears in void context, each component should have void context. This also works fine. But when a literal list appears in (not known at compile time) scalar context, every component but the last should have void context. And perl's context propagation isn't smart enough for that. Instead every component gets scalar context and a cleanup operator (named "list") ensures that in fact the list of results is reduced to the last scalar. This is quite clearly a case of a list in scalar context.

    The following code demostrates this, using a tied array as a left operand to a comma operator in each of void, scalar, and list context, first where the context is known at compile time, and then where it isn't:

    $ perl $ perl use Tie::Array; @ISA="Tie::StdArray"; sub FETCHSIZE{print "in FETCHSIZE\n";my $self=shift;$self->SUPER::FETC +HSIZE(@_)}; tie @x,"main"; @x=1..3; print "void\n"; @x,1; print "scalar\n"; $x=(@x,1); print "list\n"; ()=(@x,1); print "void\n"; sub{@x,1}->(); print "scalar\n"; $x=sub{@x,1}->(); print "list\n"; ()=sub{@x,1}->(); __END__ void scalar list in FETCHSIZE void scalar in FETCHSIZE list in FETCHSIZE
    Note that where the scalar context is known at compile time, the left operand to , got void context, but where it was not known, it got scalar context.

      I prefer to stick to models that, well, model what actually happens.

      If that works for you, then that is exactly what you should do.

      But, you are very much "in the know", about the perl internals. This allows you to have a model that reflects "what actually happens". This is clearly not the case for everyone.

      Not even those ranked amongst those who are in a position to know, given the contradiction between your statement a little further down: "This is quite clearly a case of a list in scalar context.", and that other "in the know" stance frequently repeated in this thread and it's predecessor.

      So, what value do you see in advocating this model?

      In a phrase: Conceptual simplicity.

      In an attempt to back that reasoning up, I quote a few selected passages* from Chapter 7:"Being Analog of a book, The Invisible Computer by Donald Norman a professor emeritus in cognitive science and a Professor of Computer Science.

      (*)I'd like the quote the whole chapter, and highly recommend the entire book.

      Making Sense of the World

      If an airplane crashes on the border between the United States and Canada, killing half the passengers, in which country should the survivors be buried?

      We are social creatures, understanding creatures. We try to make sense of the world. We assume that information is sensible, and we do the best we can with what we receive. This is a virtue. It makes us successful communicators, efficient and robust in going about our daily activities. It also means we can readily be tricked. It wasn't Moses who brought the animals aboard the ark, it was Noah. It isn't the survivors who should be buried, it is the casualties.

      It's a good thing we are built this way: this compliance saves us whenever the world goes awry. By making sense of the environment, by making sense of the events we encounter, we know what to attend to, what to ignore. Human attention is the limiting factor, a well known truism of psychology and of critical importance today. Human sensory systems are bombarded with far more information than can be processed in depth: some selection has to be made. Just how this selection is done has been the target of prolonged investigation by numerous cognitive scientists who have studied people's behavior when overloaded with information, by neuroscientists who have tried to follow the biological processing of sensory signals, and by a host of other investigators. I was one of them: I spent almost ten years of my research life studying the mechanisms of human attention.

      One understanding of the cognitive process of attention comes from the concept of a "conceptual model," a concept that will gain great importance in Chapter 8 when I discuss how to design technology that people can use. A conceptual model is, to put it most simply, a story that makes sense of the situation.

      I sit at my desk with a large number of sounds impinging upon me. It is an easy matter to classify the sounds. What is all that noise outside? A family must be riding their bicycles and the parents are yelling to their children. And the neighbor's dogs are barking at them, which is why my dogs started barking. Do I really know this? No. I didn't even bother to look out the window: my mind subconsciously, automatically created the story, creating a comprehensive explanation for the noises, even as I concentrated upon the computer screen.

      How do I know what really happened? I don't. I listened to the sounds and created an explanation, one that was logical, heavily dependent upon past experience with those sound patterns. It is very likely to be correct, but I don't really know.

      ...

      A good conceptual model of events allows us to classify them into those relevant and those not relevant, dramatically simplifying life: we attend to the relevant and only monitor the irrelevant. Mind you, this monitoring and classification is completely subconscious. The conscious mind is usually unaware of the process. Indeed, the whole point is to reserve the conscious mind for the critical events of the task being attended to and to suppress most of the other, non-relevant events from taking up the limited attentional resources of consciousness.

      On the whole, human consciousness avoids paying attention to the routine. Conscious processing attends to the non-routine, to the discrepancies and novelties, to things that go wrong. As a result, we are sensitive to changes in the environment, remarkably insensitive to the commonplace, the routine.

      Conceptual models do not always have to be total accurate. Indeed, they frequently benefit from not being so.

      Would you trade your own understanding of 'right and wrong', for having to know and remember the entire corpus of 'The Law' applicable to you and your actions, and to which you may be held to account. There's an axiom that goes: "Ignorance of the law is no defence.". So how can any of us hope to know every law: federal, local, commercial, moral; to which we are subject?

      And the answer is we cannot. Instead we make do with a conceptual model--for example: would we be upset if someone did this to us?, If so, don't do it to others--that we hope stands us in good stead in most situations.

      Some would advocate that this ambiguity, the very need for conceptual simplification, is reason enough to avoid reliance upon Perl's more magical features. Some would even advocate that the entire concept of context should be dropped as too complex. Preferring instead some quantified set of rules for how the language operates and how it should be used. I can only refer them to a later section, "Human Error", of Chapter 7 linked above, where it says:

      Programming languages are difficult to learn, and a large proportion of the population is incapable of learning them. Moreover, even the most skilled programmers make errors, and error finding and correction occupy a significant amount of a programming team's time and effort. Moreover, programming errors are serious. In the best circumstances, they lead to inoperable systems. In the worst, they lead to systems that appear to work but produce erroneous results.

      A person's first human language is so natural to learn that it is done without any formal instruction: people must suffer severe brain impairment to be incapable of learning language. Note that "natural" does not mean "easy": it takes ten to fifteen years to master one's native language. Second language learning can be excruciatingly difficult.

      Natural language, unlike programming language, is flexible, ambiguous, and heavily dependent on shared understanding, a shared knowledge base, and shared cultural experiences. Errors in speech are seldom important: Utterances can be interrupted, restarted, even contradicted, with little difficulty in understanding. The system makes natural language communication extremely robust.

      Natural language is full of context. It relies upon it for it's robustness. Perl (5; 6 is still an open question) approaches the level of flexibility, ambiguity and yes, "even contradiction" of natural languages more closely than any other I've knowledge of. You can pick up the basics through osmosis just as a child does with their native language, but to acquire fluency requires long term immersion.

      And that final point is why this site could never be replaced with a static corpus of POD, FAQs, and best practices. A couple more quotes:

      The United States Navy has a formal, rigid hierarchy of command and control, with two classes of workers -- enlisted crew and officers -- and a rigid layer of formal rank and assignment. There are extensive procedures for all tasks. Yet in their work habits, especially in critical operations, rank seems to be ignored and crew members frequently question the actions. Sometimes they even debate the appropriate action to be taken. The crew, moreover, is always changing. There are always new people who have not learned the ship's procedures, and even the veterans often don't have more than two or three year's experience with the ship: the Navy has a policy of rotating assignment. Sounds horrible, doesn't it? Isn't the military supposed to be the model of order and structure? But wait. Look at the outcomes: the crew functions safely and expertly in dangerous, high-stress conditions. What is happening here?

      Do we need procedures? Of course. The best procedures will mandate outcomes, not methods. Methods change: it is the outcomes we care about. Procedures must be designed with care and attention to the social, human side of the operation. Else we have the existing condition in most industries. If the procedures are followed exactly, work slows to an unacceptable level. In order to perform properly it is necessary to violate the procedures. Workers get fired for lack of efficiency, which means they are subtly, unofficially encouraged to violate the procedures. Unless something goes wrong, in which case they can be fired for failure to follow the procedures. Now look at the Navy. The apparent chaos, indecision and arguments are not what they seem to be. The apparent chaos is a carefully honed system, tested and evolved over generations, that maximizes safety and efficiency even in the face of numerous unknowns, novel circumstances, and a wide range of skills and knowledge by the crew. Having everyone participate and question the actions serves several roles simultaneously. The very ambiguity, the continual questioning and debate keeps everyone in touch with the activity, thereby providing redundant checks on the actions. This adds to the safety, for now it is likely for errors to get detected before they have caused problems. The newer crew members are learning, and the public discussions among the other crew serve as valuable training exercises, training mind you not in some artificial, abstract fashion, but in real, relevant situations where it really matters. And by not punishing people when they speak out, question, or even bring the operations to a halt, they encourage continual learning and performance enhancement. It makes for an effective, well-tuned team.

      New crew members don't have the experience of older ones. This means they are not efficient, don't always know what to do, and perform slowly. They need a lot of guidance. The system automatically provides this constant supervision and coaching, allowing people to learn on the job. At the same time, because the minds of the new crew members are not yet locked into the routines, their questioning can sometimes reveal errors: they challenge the conventional mindset, asking whether the simple explanation of events is correct. This is the best way to avoid errors of misdiagnosis.

      The continual challenge to authority goes against conventional wisdom and is certainly a violation of the traditional hierarchical management style. But it is so important to safety that the aviation industry now has special training in crew management, where the junior officers in the cockpit are encouraged to question the actions of the captain. In turn, the captain, who used to be thought of as the person in command, with full authority, never to be questioned, has had to learn to encourage crew members to question their every act. The end result may look less regular, but it is far safer.

      This place, and especially threads like this, are exactly why this place works, It is exactly this type of debate, all too often stifled, that turns drive-by questioners into long-term, active participants. (And it should be encouraged, not frowned upon.)


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
        So, what value do you see in advocating this model?
        In a phrase: Conceptual simplicity.
        That's the obvious answer. But I was hoping for more of a demonstration of benefit in the answer. Simplicity has positives and negatives, as well as various levels. On the simplest level, you can think of perl having both scalar and list values, and describe what different operations do with them. I think this encourages mistakes like $foo{bar} = (1,2,3); and @baz = [1,2,3];. The biggest reason I see for moving to a less simple (but still inaccurate!) model of "no such thing as a list in scalar context" is that I think it encourages more of an "operators provide context to their operands" pattern of thought that is the basis for really understanding what you are doing when you code perl. Of course, it would be great if everyone could shed even that level of simplicity, but that's asking a bit much.

        Nice post. Thank you for taking the time to write it.

        When this particluar discussion started, I must admit that I found it a bit ridiculous. Lists in scalar context?? What?? That just didn't make sense to me at an intuitive level. But, after following the posts here, I now have a greater insight into the way some other people have structured their programming paradigms and I expect that this insight will prove helpful to me when communicating with other programmers.

        This is the sort of discussion that sets PerlMonks apart from other programming sites and I applaud those who have taken part and cogently explained their views.

Re: If you believe in Lists in Scalar Context, Clap your Hands
by ig (Vicar) on Oct 29, 2008 at 05:53 UTC

    There appear to be two possibilities for LISTs:

    One is that expressions that look like LISTs are LISTs regardless of whether they will be evaluated in list or scalar context, but the value and the process of evaluation depends on the context, as is generally the case with Perl.

    When a LIST is evaluated in scalar context every element of the list is evaluated in scalar context and the value of the last element is the value of the LIST, with all the other values being discarded.

    The other possibility is that expressions that look like LISTs are LISTs if and only if they are being/will be evaluated in list context but they are "what apperas to be a LIST"s if they are being/will be evaluated in scalar context. Commas in the part that is a LIST or a "what appears to be a LIST" are either list argument separators (not operators) or comma operators respectively. A statement may contain a LIST or a "what appears to be a LIST" depending on the context in which it is being/will be evaluated at run time and the rules for evaluating a LIST are different from those for evaluating a "what appers to be a LIST". The LIST only exists if the expression is being evaluated in list context and the "what appears to be a LIST" only exists if the expression is being evaluated in a scalar context. Therefore, there is no possibility of a LIST being evaluated in scalar context, avoiding considerable confusion.

    When a "what appears to be a LIST" is evaluated in scalar context, by virtue of the comma operator, every element in the "what apperas to be a LIST" is evaluated in scalar context and the value of the last element is the value of the "what appears to be a LIST" with all the other values being discarded. (Note: all but the last element may be evaluated in void context rather than scalar context, but it still counts as scalar context for functions that care whether or not they're being called in list context.)

    The former model is simpler and less surprising to me. It seems entirely consistent with the general rule that the interpretation of operations and values depends on context.

    Whether any particular version of perl evaluates what appears to be a LIST in scalar context by evaluating all the elements in scalar context and using their values to initialize an ephermeral list value then discards all but the last value, or whether it has been optimized to evaluate all but the last element in void context (which is just a special case of the scalar context) and evaluate only the last element in scalar context and uses its value to initialize an ephemeral scalar value doesn't seem relevant to my understanding of Perl.

    I suggest that the definition of Perl should be a simple and unsurprising specification of what the result of executing a Perl program will be, not a detailed specification of what perl will do when executing it.

    clap

      When a LIST is evaluated in scalar context every element of the list is evaluated in scalar context and the value of the last element is the value of the LIST, with all the other values being discarded.

      That explanation fails to account for operator precedence, so it's unworkable in practice.

        Would you elaborate on the affect of precedence on this please? With an example if possible.

        That explanation fails to account for operator precedence, so it's unworkable in practice.

        I don't understand your point. Can you give an example or explain how it fails to account for operator precedence?

        Note that I am talking about LISTs, not list values or all expressions that have list values in list context. I think it is necessary to distinguish between these three quite different things and that failure to do so is at the root of some of the confusion and disagreement that has been expressed.

        Update: chromatic responded to AM about the same time I posted this.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://719099]
Approved by CardinalNumber
Front-paged by Erez
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (7)
As of 2014-10-02 08:29 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    What is your favourite meta-syntactic variable name?














    Results (52 votes), past polls