Perl Idioms Explained - && and || "Short Circuit" operators

[davido]

Inspired by some goings-on yesterday in the CB I realized that a discussion of the many uses of the && and || operators might be in order. Camel (Programming Perl, O'Reilly) calls them "C-style Logical (Short Circuit) Operators." So can something so blatently borrowed from C be considered an integral Perl idiom? Definately. Here's why, and why they're an important part of the Perl toolbelt.

C-style?
First, to set the record straight, though they are "C-style operators", they are not carbon copies of their C cousins. perlop states, "The || and && operators differ from C's in that, rather than returning 0 or 1, they return the last value evaluated." And therein lies one of the important powers of these operators. They are often referred to as being "C-style" because C is a well known language that helped to establish the custom of short-circuit behavior for these operators (and that's the last time I'll mention C... for now).

Why "Short Circuit?"

Camel II tells us that the term "Short-Circuit" refers to the fact that they "determine the truth of the statement by evaluating the fewest number of operands possible." So essentially these operators stop the chain of evaluation of an expression as early as possible. That is another key to their value.

Why "Logical?"
Logical just means that the && and || operators impose some logic on the expression they're a part of. And logic implies decision making, or conditional flow control. That's the final key to the power these operators wield.

What do they do?

---------------------------------------------------------
$this && $that   |    If $this is true, return $that,          
$this and $that  |    else return $this.
-----------------+---------------------------------------
$this || $that   |    If $this is true, return $this,
$this or $that   |    else return $that.
---------------------------------------------------------

The above table also lists && and ||'s siblings: 'and' and 'or'. The latter two work the same as the former two, except with a lower precedence, so as to reduce the need for parenthesis. More on that later.

A closer look at && logic and short-circuiting:
If you're already comfortable with how they work, skip the next two code segments and just resume reading with the discussion of actual Perl idioms.

Here is a set of simple examples that illustrate what these operators are doing:

• && with both expressions true.

  my ($first, $second) = ( 1, 1 );
  print "Truth\n" if $first++ && $second++;
  print "First: $first\nSecond: $second\n";
  __OUTPUT__
  Truth
  First: 2
  Second: 2
  

• && with first expression true and second false.

  my ($first, $second) = ( 1, 0 );
  print "Truth\n" if $first++ && $second++;
  print "First: $first\nSecond: $second\n";
  __OUTPUT__
  First: 2
  Second: 1
  # Note: Both are evaluated because the first one is
  # true.  But "Truth" isn't printed, because only one
  # of the two expressions evaluated "true".
  

• && with first expression false and 2nd true.

  my ($first, $second) = ( 0, 1 );
  print "Truth\n" if $first++ && $second++;
  print "First: $first\nSecond: $second\n";
  __OUTPUT__
  First: 1
  Second: 1
  # $second didn't get incremented because the
  # evaluation stopped when $first evaluated false.
  

• && with both expressions false.

  my ($first, $second) = ( 0, 0 );
  print "Truth\n" if $first++ && $second++;
  print "First: $first\nSecond: $second\n";
  __Output__
  First: 1
  Second: 0
  # $first was evaluated for truth.  It was false,
  # so $second didn't get evaluated.
  

In the above example you can see short-circuiting and logic both in action. "Truth" is only printed in the first example, because that is the only example where $this and $that were true at the time of evaluation. The rest of the examples showed how short-circuiting affected whether or not the second expression ($that) ever got evaluated. You can see if it gets evaluated by virtue of the ++ operator: If it got incremented, it got evaluated.

||'s version of logic and short-circuiting: The following examples express in fairly simple terms what || does:

• || with both expressions true.

  my ($first, $second) = ( 1, 1 );
  print "Truth\n" if $first++ || $second++;
  print "First: $first\nSecond: $second\n";
  __OUTPUT__
  Truth
  First: 2
  Second: 1
  # Since $first is true, no need to evaluate $second;
  # we already know that the 'or' expression is true.
  

• || with first expressions true, second false.

  my ($first, $second) = ( 1, 0 );
  print "Truth\n" if $first++ || $second++;
  print "First: $first\nSecond: $second\n";
  __OUTPUT__
  Truth
  First: 2
  Second: 0
  # Again, $second is never evaluated because $first
  # is true, and that's good enough for ||.
  

• || with first expression false, second true.

  my ($first, $second) = ( 0, 1 );
  print "Truth\n" if $first++ || $second++;
  print "First: $first\nSecond: $second\n";
  __OUTPUT__
  Truth
  First: 1
  Second: 2
  # Both sides got evaluated because since $first was
  # false it was necessary to evaluate $second to 
  # determine if truth exists (it does).
  

• || with both expressions false.

  my ($first, $second) = ( 0, 0 );
  print "Truth\n" if $first++ || $second++;
  print "First: $first\nSecond: $second\n";
  __OUTPUT__
  First: 1
  Second: 1
  # There is no truth.  Both expressions were
  # evaluated to find it.
  

So to use a catchy phrase, && "falls through" if the first expression is true, and returns truth if both are true.
|| "falls through" if the first expression is false, and returns truth if either expression is true.
And as mentioned before "returns truth" means returns the last expression evaluated.

So what about the idioms?
There are a lot of them. Here are a few to whet your apetite:

open( FILE, "<filename" ) || die "Cannot open filename: $!\n";
# Open the file.  If the open fails (and thus evaluates 
# false) fall through to the second half: die.

That's one you see all over the place. In a moment I'll mention why "open(...) or die ..." is better than "open(...) || die ...". The great thing about these operators is that they "read well". They do exactly what they say: "open or die", for example.

Here's another common idiom that uses && to provide a C-like "switch" statement (Ok, now I promise not to mention C anymore):

local $_ = 'xyz';
SWITCH: {
    /^abc/ && do { $abc = 1; last SWITCH; };
    /^def/ && do { $def = 1; last SWITCH; };
    /^xyz/ && do { $xyz = 1; last SWITCH; };
    $default = 1;
}

(Credit to perlsyn for the bulk of that example.) The preceeding example may look a little funky. How does it make sense to say "if this and do that"? Remember that "do" returns the value of the last expression evaluated inside it. So basically the preceeding example says "If this and that." But the truthfulness of the second expression isn't important anyway. What we're doing is evaluating the truthfulness of the first expression to determine whether or not to fall through to evaluate the second expression. So in the preceeding example, "$_ =~ /^xyz/" is true, so everything inside the do{ ... } expression gets evaluated, or executed. This is one of the ways to implement switch (hashtables can be more efficient though). Now you know one reason why Perl 5 doesn't have a switch statement (though there is a Switch module if you really need switch, and Perl 6 will have it).

You are free to chain these operators together as well. You could, for example, say:

if ( $this && $that && $other ) { print "Truth\n"; }

or even...

my $client = $ENV{USER_HOST} ||
             $ENV{USER_ADDR} ||
             "UNKNOWN";

And now for the super-cool idiom that reminded me that I wanted to write this node in the first place:

my @sorted = sort { uc($a) cmp uc($b) || $a cmp $b } @unsorted;

What is going on here? Remember that "cmp" and "<=>" evaluate to "zero" (false) if the left hand side and the right hand side are equal, and "false" will make '||' fall through to the next level. We're sorting a list case-insensitively, but if there is both an upper-case and lower-case string that, when evaluated case-insensitively, would evaluate to being equal, the || will fall through to the second expression, which evaluates case-SENsitively. In other words: (qw/b c a C D E e f/) would sort as: (qw/a b C c D E e f/) (assuming you're on an ASCII system).

This idiom can be wielded in an even more powerful way. Consider, for example, a multi-dimensional entity where you want to sort complexly (is that a word?):

my @list = ( { 'Name' => "Pete", 'Age' => 32 },
             { 'Name' => "Pete", 'Age' => 55 } );
my @sorted = sort { $a->{'Name'} cmp $b->{'Name'} ||
                    $a->{'Age' } <=> $b->{'Age' }   } @list;

In that example, you're going to sort by name first, and whenever two names are equal, fall through to subsort by age.

Precedence
There are situations where && and || may be of too-high precedence, thus necessitating the use of parenthesis. The "open or die" example provided above is one of those situations. But && and || have younger siblings with lower precedence. The "open or die" example could be written another way:

open ( FILE, "<filename" ) || die ..... # You already saw this one.
open FILE, "<filename" or die .... # Notice how the much lower
# precedence of "or" makes the parenthesis unnecessary in
# this case.

Just think of "and" and "or" as being the same as && and ||, but so low on the precedence chart that parenthesis are usually unnecessary to keep the things on the left or on the right properly grouped. It is almost always considered better to use "or" instead of "||" when you work with open.

Ok, that's all I have to say. Thanks for staying awake through this longwinded node!

I hope some readers find this discussion helpful.

Update: My thanks to hardburn, Abigail-II, tye, Enlil, diotalevi, demerphq and a few others for providing comments that helped me to refine this node a bit.

Additional Reading:
Programming Perl O'Reilly: Chapter on Operators.
perlop from the Perl POD.
perlsyn from the Perl POD.
perlfunc: sections on sort and open.
perlopentut: tutorial on opening files.

[Abigail-II]

---------------------------------------------------------
$this && $that   |    If $this is true, and $that is true
                 |    return the value of the last
                 |    expression evaluated ($that).
-----------------+---------------------------------------
$this || $that   |    If $this evaluates true, stop 
                 |    evaluating, we're done.  Otherwise
                 |    evaluate $that for truthfulness.
                 |    If $that evaluates true, return 
                 |    the value of $that.
---------------------------------------------------------

That doesn't explain what is returned if $that is false. May I suggest the following (which is shorter, symmetric, and explains the return value on failure).

---------------------------------------------------------
$this && $that   |    If $this is true, return $that,          
                 |    else return $this.
-----------------+---------------------------------------
$this || $that   |    If $this is true, return $this,
                 |    else return $that.
---------------------------------------------------------

Abigail

[davido]

Thanks Abigail. The node has been updated per your suggestion. Good catch.

[hardburn]

Now you know why Perl doesn't have a switch statement . . .

It should still get one, since the advantage of C switches it that they have O(1) efficency, whereas most of the equivilent idioms in Perl have O(n) worst-case time. <UPDATE>Actually, the one posted above always runs in O(n) time.</UPDATE> <UPDATE2>Oops, that update wasn't quite true. Thanks demerphq.</UPDATE2> You can get O(1) time using a hash table that stores subroutine refs, but then you make implementing fall-through a lot harder.

Fortunatly, we're getting a real switch in Perl6.

[demerphq]

Im very confused about this node. How does C's switch statement offer O(1) time? Im wracking my brain to see how this could be done without becoming hopelessly inefficient. Also your update is confusing too, the switch like code posted drops out of the loop once a condition has been met. I suppose strictly speaking a series of conditionals can be described as O(N), but this deosnt seem to match up with the spirit of your usage.

[fletcher_the_dog]

C switch statements create an internal jump table that is more or less like a hash so that they can skip testing cases that are are going to fail, this makes them O(1). For example in the following switch statement if 'test' equals 'c' the code jumps right to case 'c', it doesn't test to see if 'test' equals 'a' or 'b':

switch(test){
  case 'a':
   # do something
  case 'b':
   # do something else
  case 'c':
   # do something else
}

This of course means that the possible cases (but not the test value) are static and must be known at compile time which decreases flexibilty but increases speed.

[pg]

It is correct to say C's switch is O(1).

Theoritically speaking, BrowserUK is also wrong when he said O(1) means test once. However he had something in () said "in this case", which made himself "politically" right ;-)

Strictly speaking, O(1) means that the cost is a constant, not a function of any variable, thus it is considered ideal, as the cost is 100% predictable. The complexity (worst scenario cost) of the switch statement is determined at the coding time, not base on the input data at execution time.

C's switch statement falls into this category. I personally believe that there is a benefit to have it in Perl.

[BrowserUk]

Now you've confused me:)

Doesn't O(1) (in this case) mean that a condition is only tested once with the C-switch statement. Whereas the implementation shown in the OP, all tests from 0 to N where N is the met condition's lexical position within the group, hence worst case O(N) if the last case is the one chosen?

It was this bit of your post that confused me

...Im wracking my brain to see how this could be done without becoming hopelessly inefficient....

---

Examine what is said, not who speaks.
"Efficiency is intelligent laziness." -David Dunham
"Think for yourself!" - Abigail
Hooray!

[demerphq]

C-style?

You split "C-style" and "short cicuit" into two sections. The whole point of refering to them as "C-Style" is because they are "short circuit". I believe the reason was that C is the most commonly known old school language that defines its logical operators to "short circuit". Turbo Pascal's logical operators were short circuit, but I dont know if this is part of the Pascal language definition, or just one of many Borland extensions. OTOH, Basic generally does not have short circuit operators (and thus presumably Fortran does not have them either). However I'd guess that most modern languages have operators which short circuit considering the convenience they offer. Without them you have to write this

if (defined $array->[1] and $array->[1] eq 'foo') {
  ...
}

as a nested if, like this

if (defined $array->[1]) {
  if ($array->[1] eq 'foo') {
    ...
  }
}

which is a source of perpetual frustration to me when I have to hack on VB code. (A chore that luckily I do very rarely these days.)

Anyway, good meditation.

[Zaxo]

It's worth mentioning that the low precedence operators and and or work the same way, differing from && and || only in their precedence. The low precedence is useful in common idioms like parenthesis-free my $pid = open my $fh, '-|', '/path/to/cmd' or die $!;.

[decnartne]

Pardon the perhaps obtuse questions, but

1.) why *exactly* is "It almost always considered better to use "or" instead of "||" when you work with open" and
2.) when considering  open( FH, "file" ) or die ... is there an error by including parens? in idiom? in style?

the asbestos suit is on, so flame away ;)

[davido]

I never intended to say there's something stylistically or idiomatically wrong with using parens with the open (or any other) function. perlstyle states that it is stylistically good to "Omit redundant punctuation as long as clarity doesn't suffer." But a few paragraphs later says, "...just because you CAN omit parenthesis in many places doesn't mean you should." But the example given is a situation with many levels of nested functions.

perlop has this to say by way of introduction to "or":

As more readable alternatives to && and || when used for control flow, Perl provides and and or operators (see below). The short-circuit behavior is identical. The precedence of "and" and "or" is much lower, however, so that you can safely use them after a list operator without the need for parentheses...

Here is a key section of perlop:

Binary "or" returns the logical disjunction of the two surrounding expressions. It's equivalent to || except for the very low precedence. This makes it useful for control flow:

print FH $data  or   die "Can't write to FH: $!";

This means that it short-circuits: i.e., the right expression is evaluated only if the left expression is false. Due to its precedence, you should probably avoid using this for assignment, only for control flow.

    $a = $b or $c;     # bug: this is wrong
    ($a = $b) or $c;   # really means this
    $a = $b || $c;     # better written this way

However, when it's a list-context assignment and you're trying to use "||" for control flow, you probably need "or" so that the assignment takes higher precedence.

    @info = stat($file) || die;    # oops, scalar sense of stat!
    @info = stat($file) or die;    # better, now @info gets its due

Then again, you could always use parentheses.

perlopentut says:

If you prefer the low-punctuation version, you could write that this way:

    open INFO, "< datafile"  or die "can't open datafile: $!";

perlfunc open recommends 'or' but then shows examples with both 'or' and '||'.

My own reason for asserting it is amost always considered better is for several reasons: First, I can see the logic of it; it allows you to not worry about whether you have parenthesis in place. Second, it reads better; 'do this or that' makes for clearer reading than 'do this || that'. Third, after posting the original node, I received comments by several people reminding me that I should increase my emphasis on using 'or' in favor of '||' with open and in other flow-control places because it reduces the ambiguity of precedence. There is no question that "or" is going to be near the bottom of the precedence ladder, whereas "||" is quite a bit higher, and thus, one has to remember when constructing complex logical short-circuit operations which operators are going to bind more closely and which are going to bind less tightly.

Perhaps "almost always" is too generous. But I do believe it is clearer, cleaner, less ambiguous, and less prone to context problems. 'or' probably shouldn't be used for "assignment" purposes (the examples above show this). But it seems to be a good choice for many logical flow needs.

[decnartne]

wow... thanks for such a thorough treatment of my questions in your reply and for the parent as well.