Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery

On Scalar Context

by merlyn (Sage)
on Apr 22, 2004 at 17:10 UTC ( [id://347416] : perlmeditation . print w/replies, xml ) Need Help??

In response to a P5P posting, I strolled through perlfunc and came up with a representative list of the scalar context return values for things we normally use in a list context. Surprisingly, more things don't return a length than do. Here's the list:
@foolength (efficiently)
keys/valueslength (efficiently)
splitlength, but @_ side-effect
(10, 20, 30)last element
@foo[3..5]last element
(10, 20, 30)[2, 1]last element
splicelast element
callerpackage name (first element of list return)
eachkey (first element)
getpwuidusername (first element)
getpwnamuser ID (third element of list return)
glob/<*>"next" item (repeat until undef)
gmtime/localtimeprintable string (instead of list of components)
readline/<HANDLE>"next" item (repeat until undef)
readpipe/``/qxone item instead of list of lines
readdir"next" item (repeat until undef)
reversestring reverse instead of list reverse
select(4-arg)$nfound (first element of list)
statsuccess value
times$user (first element)
unpackfirst element

-- Randal L. Schwartz, Perl hacker
Be sure to read my standard disclaimer if this is a reply.

Replies are listed 'Best First'.
Re: On Scalar Context
by davido (Cardinal) on Apr 23, 2004 at 03:08 UTC
    After reading flyingmoose's reaction to merlyn's post, I wandered through the POD myself to see if any of the behaviors that merlyn discussed are in some way undocumented such that they should come as a surprise. In just about every case, I found what I was looking for. I thought I might as well offer a little more detail on the ones that seemed most interesting.

    • grep: The POD states that in scalar context grep returns the number of times the expression was true. The fact that it is true as many times as the length of the list-context return value is ... well, expected.

    • sort: The POD for sort states that in scalar context the behavior of sort is undefined (which is different from undef. So if merlyn's experience is that it returns undef in scalar context, that behavior can't be relied upon to not change in future Perl implementations.

    • getpwnam: The POD for getpwnam is a little more generic, applying to this entire group of functions. Therefore, its description is a little vague. It states that "in scalar context you get the name, unless the function was a lookup by name, in which case you get the other thing, whatever it is." The example given is $uid = getpwnam($name); I'd say that behavior is documented though a little obscure.

    • select: The POD for select doesn't textually discuss the scalar-context return value of the four arg version of the function. However, there is an example given in the POD, and it is in keeping with merlyn's findings. Update: It is also discussed briefly in the text (thanks ambrus... good catch).

    • caller: The POD for caller states "In scalar context, returns the caller's package name if there is a caller, that is, if we're in a subroutine or eval, or require, and the undefined value otherwise." (undef)

    Regarding flyingmoose's dismay over the situation, and his call for improved consistancy, this came up awhile back in What should be returned in scalar context?. I think that the general consensus was that functions should return what it makes sense for them to return in scalar context. In other words, don't get too caught up in trying to make all list-oriented functions return the same thing in scalar context. Not every shoe fits every foot.

    To call these behaviors bugs is irrational, since they, in most cases, are doing something useful, and something that's defined in the documentation. The suggestion to not make assumptions is spot-on; that's why the POD is there -- so assumptions don't have to be made.


      I agree with davido, I don't think it makes sense to necessarily return the same thing every time in scalar context. It wouldn't make sense to have the "wantarray" function at all if scalar context always returned the length of the list. A sub-routine could just always return a list and if it was in scalar context the list length would be given. But the usefullness of "wantarray" is that you can return something different if you know you are in scalar context and not list context.
        From a evil-software design overlord perspective, I don't believe in wantarray being a good idea at all -- but you are probably glad that I'm not in charge. Technically, a function could do something entirely different based on context, not just in return type, and should it really be able to do that...

        Perl says do whatever you want, but in the general case, that's asking for trouble. Feel free to disagree, that's fine, but there are many aspects about Perl that make it sloppy. Sometimes these are the same reasons that make it flexible and cool, but sometimes not.

        So if people can argue Perl OO is ugly (most would agree), I can argue that wantarray is ugly... Never mind that wantarray seems to imply a true/false value from a design standpoint but actually returns undef, true, or false -- it's a three way statement.

      The perlfunc pod does define the return value of the 4-arg-select:

      Most systems do not bother to return anything use- ful in $timeleft, so calling select() in scalar context just returns $nfound.

      This is from the perlfunc manpage of perl 5.8.1.

Re: On Scalar Context ("list")
by tye (Sage) on Apr 23, 2004 at 18:14 UTC

    Thanks, merlyn++.

    Note that in the following cases, "last element" doesn't mean the same thing each time.

    (10, 20, 30)last element
    @foo[3..5]last element
    (10, 20, 30)[2, 1]last element

    I'd actually change one entry:

    (10, 20, 30)last expression

    Because it isn't the "last element" of the "list of scalar values that this operation would produce if called in a list context" but is instead the "last element" in the "list of Perl expressions that are separated by commas". Likewise, I might change the other two to "last scalar value" to be clearer.

    This is another perfect example of why "list" doesn't just mean one thing when talking about Perl. Each of those cases return the "last element" of some "list", it is just that "list" means two different things in those two cases.

    I think this is a big part of why people get confused and people argue when talking about "lists" in Perl. Since "list context", "scalar context", and "scalar" all have precise definitions in Perl, it is natural to think that "list" has a precise definition as well.

    #!/usr/bin/perl -wl use strict; my @idx= ( 2, 99, 5 ); my @a2z= ( 'a'..'z' ); my @arr= ( 10..16 ); print '(@arr,@a2z): ', scalar( (@arr,@a2z) ); print '(@arr,@a2z)[@idx]: ', scalar( (@arr,@a2z)[@idx] ); print '@arr[@idx]: ', scalar( @arr[@idx] );


    Useless use of private array in void context at line 9. (@arr,@a2z): 26 (@arr,@a2z)[@idx]: 15 @arr[@idx]: 15

    So only the first item is lazy (passing the scalar context inside such that a "list of scalar values on the stack" doesn't get generated just to be mostly thrown away).

    I still believe that the other two "should" have been lazy (pass their scalar context into @idx) but instead the developers of Perl were lazy and just wrote code to generate the list of scalar values and simply pick the last one at the last minute. But since slices in scalar context are more "edge" or "corner" cases, being lazy with the design was probably the right choice. (:

    So $s = (10,20,30) does contain "a list in scalar context". Either one has to admit that or one has to not say that $s gets set the the "last element of the list". :)

    - tye        

      Although it gets a bit hairy, one can explain that behavior by saying that my $a = (10,20,30); is merely an application of the comma operator in scalar context, whereas the rhs of my $a = (10,20,30)[0..2] is really a list, and so the behavior of taking the last value has to be explicitly defined. The distinction is important because I believe that  my $a = (1,2,foo) would call foo in scalar context, but my $a = (1,2,foo)[0..2] would call it in list context.

      The preceding paragraph will be moot someday because according to Apocalypse 3, there will be no scalar comma operator in Perl 6 - it will always be a list, and a list in scalar context will return an array reference

Re: On Scalar Context
by dragonchild (Archbishop) on Apr 22, 2004 at 17:57 UTC
    Cause I'm too lazy to benchmark it, would sort short-circuit and not perform the sort if called in a scalar context? What about a void context?

    We are the carpenters and bricklayers of the Information Age.

    Then there are Damian modules.... *sigh* ... that's not about being less-lazy -- that's about being on some really good drugs -- you know, there is no spoon. - flyingmoose

Re: On Scalar Context
by ambrus (Abbot) on Apr 24, 2004 at 21:11 UTC

    You've left out an important case from the table: regular expressions.

    Here's how they work wrt contexts (correct me where I'm wrong).

    Regexps without the /g flag return true/false in saclar context (the list of captures or 1 if there are no captures, empty list when does not match).

    Regexps with the /g flag match only once in scalar (or void) context and return true/false. (In list context, they match as many times as possible.)

    Substitutions (s///) without the /g flag return true/false in either context, wit the g flag they return the number of successful matches in either context.

    Also missing are the .. and ... operators which do quite different thing in scalar context than in list context.

    List assignments (of form (LLIST)=EXPR) return the length of the right hand side.

    As for your claim that less expressions return the length than not, it was quite obvious for me, as most expressions always return a 1 long list and the same one element in scalar context, all arithmetic operators and also the and, or operators work like this.

    (Also there's this interesting issue about what a substr-assignment returns, but it's the same in scalar and list contexts, so it's unrelated to this thread.)

Re: On Scalar Context
by flyingmoose (Priest) on Apr 22, 2004 at 19:14 UTC
    Wow. Most of these sound like defects even if usage in scalar index seems of dubious value and bad style. I don't see how defense of these modes can be rationalized (except not to break old stuff). Meanwhile, never make any assumptions about a list function in scalar context -- it's all random :)

    How do we move to get this standardized and rational? Any hope for Perl6 ( aka 2010? :) )

      How do we move to get this standardized and rational?

      This is standardized and rational. Its just not applying the (incorrect) naive assumption that calling a subroutine in a given context is like working with an array, hash, or list. Its standardized and rational in that most people would consider the behaviour of most of these items to be perfectly natural and DWIM.

      Think about it: What use would each() be if it returned 2 (as an array would) or the value (as a list would). Neither of these is particularly DWIM, and would mean that the language was poorly huffman coded in the sense that youd have to write my ($key)=each %hash; instead of my $key=each %hash; and in my experience wanting the keys of a hash is more common than wanting the values.

      Work through perl and youll find that all sorts of stuff has been specifically selected to be DWIM and also more or less huffman coded, the more common keywords are short, the less common ones are longer than their more commonly used relatives (my -> our, push/pop -> splice come to mind, im sure there more). On a few occassions Larry has mentioned in his writing being motivated by brevity, at least in part, in his design methodology.

      Meanwhile, never make any assumptions about a list function in scalar context -- it's all random :)

      Not random, DWIM (hopefully). But yes, never make any assumptions about the relationship between what a function returns in list or scalar context, or for that matter what it will do in void context. Theres no logic to it, beyond what the author thought was the smartest thing at the time. The point is always check the docs, or the source to see what actually does happen.

      If you read p5p you'll find numerous large threads debating how subs can have new behaviour provided for contexts where it isnt already well defined. Somebody recently said something like "the problem with defining the new behaviour is that often the ideas proposed are so far from what the people who actually might do the work to implement the feature would expect that they arent interested in doing so." Which is an interesting point in itself.


        First they ignore you, then they laugh at you, then they fight you, then you win.
        -- Gandhi

        This is a problem with the Perl mindset. DWIM has no value when "Do What I Mean" means different things to different people. Hence, there should be well defined concepts mandating that methods that normally output an array shouldn't do things that vary behavior based on the LHS. Most languages don't have return-type overloading with the same arguments, and I consider this a good thing. Perl, somehow, thinks it is different. I love Perl, but I don't believe in covering for it's faults -- or saying that the faults are really DWIMerry or features.

      Most of the functions we're speaking in this node (sort, keys, map, stat) just don't have mush sense in scalar sense. I mean, perl is designed (?) so that you can not get caught by calling a function in scalar context when you'd mean a list context.

      Suppose for example that you want to use stat. You know what it does, so you expect an array from it. As you want to use some specific entries of the array, you won't write code like $permissions= stat $filename; that just means nothing. If you use something as a list, you automatically call it in array context.

      Compare this with the other way. Calling a function that you expect to return one single value IS a possible trap, for example, you might accidentally write

      warn "customer arrived at ", localtime;
      and expect that localtime returns a timestamp like "Thu May 13 20:26:09 2004", as it does in scalar context. This kind of trap is really dangerous, I think anyone learning Perl has fallen to it at least once. (The most dangerous are operations that have entirely different side effects in different contexts, like /g regexps.)

      So, returning to the list functions, some of these have no meaning in scalar context (like sort, @arr[@inds], map), and Perl typically gives a warning whe you use them that way. Others are typically used in list context (grep, @array, stat, keys) so you would only use them in scalar context if you already expect some sort of behavior (that is, you have read the manual). Others are both used in list and scalar contexts equally often, like readline, readdir, m//g, the comma operator, the yadda-yadda operator; these really perform two different things in the two contexts. Yes, you have to know this type, but still, you won't probably get trapped by excepting something else as a scalar value if you only know the list value, only the other way round. (Well, maybe with m//g, yes.)

Re: On Scalar Context
by Aristotle (Chancellor) on Oct 02, 2005 at 06:15 UTC

    This just came up again on use Perl (and again). I see four large factions here:

    1. List functions

      Things that take a list and return a list in list context and return a count of items in scalar context. The sole notable exception is sort.

    2. Iterators

      Things that take a scalar and return a list in list context tend to return a scalar in scalar context, and you can call them with the same input scalar to get the next element of what would have been a list in list context.

    3. Literal list operators

      Things that construct a list and give it to you in list context but only its last element in scalar context.

    4. “I’m chatty in list context”

      Things that can provide a bunch of information, which you get all of in list context, and the first first bit of in scalar context.

    The rest is a tiny inconsistent minority.

    The hash-related functions kind of belong to the list functions and kind of to the iterators – particularly the oddball each, which is the only thing that functions as an iterator in both contexts. But that is because hashes are kind of oddball anyway in terms of context.

    So the return values are consistent, their consistency is just inconsistent.

    Makeshifts last the longest.