Beefy Boxes and Bandwidth Generously Provided by pair Networks
good chemistry is complicated,
and a little bit messy -LW

Auto Increment "magic" Inquiry

by brusimm (Pilgrim)
on Jan 04, 2007 at 20:54 UTC ( #593014=perlquestion: print w/replies, xml ) Need Help??
brusimm has asked for the wisdom of the Perl Monks concerning the following question:

After a bit of looking around, I have a conflict of logic I hope we can resolve!

A: The below script was an experiment I put together. I am very NEW to Perl, & I was trying to understand the auto-increment function, and I now DO understand it. (Then I got carried away)

B: After a bit of research on Perldoc & Perlmonks, I discovered that ďPerl does not define when the variable is incremented, or de-incrementedĒ AND there are no guarantees as to the consistency of the output from one system to another. (Thatís the sanity in me speaking.)

C: Barring the previous statement that this should be an unpredictable script, I have a colleague at work that can read any of these lines, and predict to me what the output will be. (The sanity just left the building again.) This prompted me try to understand the order of operations happening here. She explained it to me, and it made sense, but the next day, I canít remember for the life of me what she said. One other item she explained was the ++$i was performed on first, then $i++. Much like multiplication before addition, as an example.

D: The script below prints out, 0 to 10. (My system is an Intel, XP based system, using ActivePerl 5.8.8, build 819) in this environment, and also under Cygwin on the same system. I am not looking to make use of this script in any way, Iím just trying to understand something here, if it is to be understood. (IE: Itís magic).

Q: How is the math being done in as much as what order preference the script is using to work on the variables to do the math and how is it being applied to create the output Iíve been seeing? I tried creating a visual matrix to better understand this, but I run into some roadblocks, mental. {I hope I have asked this in a way that makes sense.}

Thank you to anyone who can help me with what little sanity I can possibly retain.

$i = $i++; # #print "\n"; # This is a noop.. effectively being 0. print "Test 0: $i\n"; $i = 0; # $i = ++$i; # print "Test 1: $i\n"; # ++ adds 1, then $i is looked at, and ++ add +ed to it. $i = 0; # $i = $i++ + ++$i; # print "Test 2: $i\n"; $i = 0; $i = ++$i + $i++; # print "Test 3: $i\n"; # $i = 0; # $i = ++$i + ++$i; # print "Test 4: $i\n"; # $i = 0; # $i = ++$i + $i++ + $i++; # print "Test 5: $i\n"; # $i = 0; # $i = ++$i + $i++ + ++$i; # print "Test 6: $i\n"; # $i = 0; # $i = ++$i + ++$i + ++$i; # print "Test 7: $i\n"; # $i = 0; # $i = $i++ + $i++ + ++$i + ++$i ; # print "Test 8: $i\n"; # $i = 0; # $i = $i++ + ++$i + ++$i + ++$i ; # print "Test 9: $i\n"; # $i = 0; # $i = ++$i + ++$i + ++$i + $i++ ; # print "Test 10: $i\n"; # print "\n";

Replies are listed 'Best First'.
Re: Auto Increment "magic" Inquiry
by chargrill (Parson) on Jan 04, 2007 at 22:23 UTC

    Maybe this can help shed some light on previous discussions here: Quantum Weirdness and the Increment Operator. I just ran across this thread the other day, in fact.

    s**lil*; $*=join'',sort split q**; s;.*;grr; &&s+(.(.)).+$2$1+; $; = qq-$_-;s,.*,ahc,;$,.=chop for split q,,,reverse;print for($,,$;,$*,$/)
Re: Auto Increment "magic" Inquiry
by GrandFather (Sage) on Jan 04, 2007 at 22:16 UTC

    Point B is the key.

    Point C is tricky. The assertion that "++$i was performed on first, then $i++" is misleading. The distinction would only be relevant for ++$i++ which is illegal in any case. In the expression $i++ + --$i neither Perl nor C make any promises about the order in which the two sub expressions are evaluated. The second paragraph of Section A7 in "The C Programming Language (second edition)" (the K&R C book) makes that absolutely clear in the case of C (and assists in understanding the issue in Perl):

    "... unless the definition of an operator guarantees that its operands are evaluated in a particular order, the implementation is free to evaluate operands in any order."

    On the basis of that statement alone any of your tests involving addition are spurious. In particular note (from the same paragraph as cited above):

    "... the order of evaluation of expressions is, with certain exceptions, undefined, even if the subexpressions involve side effects." (emphasis added)

    Your colleague is either lucky or experienced with the particular implementation you are using. Watch out for changes between versions, platforms, C compiler and compiler options - one day you are going to get bit, and it will be particuarly nasty to diagnose!

    Update: clarified the absolute clarity

    DWIM is Perl's answer to Gödel
Re: Auto Increment "magic" Inquiry
by Solo (Deacon) on Jan 04, 2007 at 21:11 UTC
    Note that just as in C, Perl doesn't define when the variable is incremented or decremented. You just know it will be done sometime before or after the value is returned.

    As I read it, the doc is telling you not to rely on the behavior of your example to be the correct and consistent behavoir, though it may be consistent on your platform.

    Another way to interpret it is that Perl uses the same mechanism C uses to perform this op, so however C behaves on your system, Perl should behave the same, but we still won't guarantee that. Thanks to chromatic for correcting this below.

    $i = $i++ + ++$i;

    Or are you looking for some explanation like: "$i begins as 0, the pre-increment comes first, making $i=1, then 0+1 is assigned to $i, then $i is post-incremented yielding 2; or is it 1+1 assigned to $i"? But I think that's exactly what the pod is telling you not to try to explain and not to use.


    You said you wanted to be around when I made a mistake; well, this could be it, sweetheart.
      Another way to interpret it is that Perl uses the same mechanism C uses to perform this op, so however C behaves on your system, Perl should behave the same...

      Perl doesn't use the same mechanism as C, however. Perl uses an optree, which (compiled) C doesn't do. (A C compiler may use a similar tree structure for optimizations, but it doesn't guarantee the order of operations for similar reasons, not because it uses the same mechanism.)

      Solo, The second part of your post is what I am attempting to discover here. The explanation.

      Since my co-worker can seem to accurately predict the printed results, it seems to defy everything (Or is it pure luck?) I've read in the reference materials that I quoted in my posting, that are available online. That's why I'm thinking someone may have an insight as to how this can happen on my system in such a consistent fashion. If not, so be it, but I was hoping to at least have a better understanding of it on my system, as it's acting.

      Thank you Solo


        It's consistent because Perl will generate the code the same way each time you compile it. However, another version of Perl may generate slightly different code for the same sequence.

        For example, suppose you have the statement $j = $i++ + $i++;. It's perfectly acceptable for perl to implement it as:

        $temp_1 = $i; $temp_2 = $i; $j = $temp_1 + $temp_2; $i = $i + 1; $i = $i + 1;
        and it's also acceptable for it to implement it like:

        $temp_1 = $i; $i = $i + 1; $temp_2 = $i; $i = $i + 1; $j = $temp_1 + $temp_2;
        (Other implementations are also possible...) So while any particular perl compiler will be consistent from run to run, and program to program, different perl compilers (version, platform, vendor, etc.) are free to do it another way.

        Your co-worker appears to know which way it's implemented on your perl compiler.


        Things can be entirely consistent and predictable and yet not reliable.


        When the docs say you can't rely on something, it means that the behaviour isn't part of the public interface. It's liable to change without notice.

        In fact, docs are more likely to say that about predictable things, since those are the very behaviours which users are likely to assume are reliable.

        This sort of thing is more of a problem when you do too much learning through experimenting. You can't tell whether things you learn via experimentation are defined behaviour or an accidental side-effect of implementation, so you need to check them against the docs before relying on them.

Re: Auto Increment "magic" Inquiry
by blokhead (Monsignor) on Jan 04, 2007 at 22:17 UTC
    <Update> After writing this, I realize that it makes no difference that ++$i returns an alias. But I'm too lazy to rewrite the whole thing. The main point is the order in which you evaluate the arguments to the addition operation in those big expressions.</Update> Update2: changed my example so that aliases *did* make a difference. Thanks ysth++.

    Statements like yours (with multiple assignments to $i in a single statement) are confusing/unpredictable because there are aliases involved. Preincrement ++$i returns an alias to $i, while postincrement $i++ returns just a plain old value. You can see that this is the case:

    sub foo { print \$_[0], $/ } foo($i); # SCALAR(0x81526f0) foo(++$i); # SCALAR(0x81526f0) foo($i++) # SCALAR(0x8151c28)
    Now that you know aliases are involved, it's a simple matter of just following the execution.. No magic involved.

    So let's look an example:

    $i = 0; $i = ++$i + ++$i + $i++ + ++$i;
    First, recall that addition is left-associative. Also, check that when evaluating an addition, perl evaluates the left argument before the right argument. At least, that's how it happens in my perl. If there is anything "undefined" about your test cases, it is this part, not the fact that you are using increment operators. So here's how perl evaluates this statement:

    value of $iexpressionnext thing to evaluate
    0((++$i + ++$i) + $i++) + ++$i First ++$i sets $i to 1, returns alias to $i
    1((alias + ++$i) + $i++) + ++$i Second ++$i sets $i to 2, returns alias to $i
    2((alias + alias) + $i++) + ++$i contents of $i + contents of $i = 4
    2(     4         + $i++) + ++$i Third $i++ returns 2, sets $i to 3
    3(     4       + 2) + ++$i 4 + 2 = 6
    3                6        + ++$i Fourth ++$i sets $i to 4, returns alias
    4                6        + alias 6 + contents of $i = 10
    4                         10

    Now hopefully you agree with what everyone has been saying, that assigning to a variable multiple times while using its value in the same statement is really not worth it. But I wouldn't say that it's black magic, once you understand about aliasing.

    It's up to you to decide it perl will ever do anything other than left-to-right evaluation of arguments to the addition operator. I guess if perl wants to reserve the right to do something else, you'll have to respect that.


      After writing this, I realize that it makes no difference that ++$i returns an alias. But I'm too lazy to rewrite the whole thing. The main point is the order in which you evaluate the arguments to the addition operation in those big expressions.
      It does make a difference, but not in the example you show; to see a difference, you need to have something modify $i before the alias is used, like ++$i + $i++.
Re: Auto Increment "magic" Inquiry
by jettero (Monsignor) on Jan 04, 2007 at 21:04 UTC

    I think this stuff is covered rather well in perlop. I don't mean to sound flippant or anything. If you find the precedence confusing, it's probably best to use lots of parentheses to make it more obvious to the compiler what it is that you expect.


      This has little or nothing to do with order of precedence. It has to do with the fact that when side effects happen is undefined. That means that, regardless of any parenthesization, any expression that uses a variable with autoincrement more than once likely has multiple possible acceptable values.

      Consider $i++ + $i
      The 2nd $i could be either the incremented value or the unincremented value. Parentheses won't change that, because the increment happens at some undefined time. The value of $i++ is defined as the unincremented value, but if you use $i elsewhere in the same statement, Perl does not guarantee you what its value will be.

      Caution: Contents may have been coded under pressure.

        It's not just order of precedence though, it's that left right shift parsing stuff that lex/yacc/bison/etc do. It makes little sense to me, but that's really what I was talking about. And I'm pretty sure perlop lists the operations as assoc or left — which I assumed was referring to the operator shifting in the parser generator. It hardly matters since there's general agreement that the operations are unreliable. The correct solution is to split up the line (in my opinion).

        I wish I had been more clear in the first place. And I wish I'd said split up the line instead of talking incoherently about parens. I actually still kinda wonder if you can get everything to happen in the same order reliably with parens, but it seems not.


Re: Auto Increment "magic" Inquiry
by brusimm (Pilgrim) on Jan 05, 2007 at 17:37 UTC
    Wow, where do I start?

    jettero, Any opinion gives me a different perspective to consider. Thanks. I did look around as much as I could, and came up with the quoted statements in my posting. (Though, later on, it is noted there is some other "coverage" on the issue.) Roy Johnson & Solo, I am not relying on the behavior for anything of serious nature.

    roboticus, my co-worker is.. "unique" in this scenario. Her Perl is of a different flavor, and I just tested this script on a unix system, using a different version of Perl, with the same results. (I AM NOT JUSTIFYING my results, just adding to my observations.)

    jbert, you just hurt my brain with: Things can be entirely consistent and predictable and yet not reliable. That's like being in a Rigid State of Flexibility!! All joking aside, I did try to uncover supporting documentation, but alas, there is none, for understandable reasons.

    GrandFather, I' won't get "bit" because I do not have any plans to ever try to implement this kind of unstable insanity!.. (Was that redundant? Did I just create a variable making sanity, sane or what?). I'm just experimenting, trying to understand things, get to the bottom of my own curiosities.

    blokhead "Now hopefully you agree with what everyone has been saying, that assigning to a variable multiple times while using its value in the same statement is really not worth it." Oh yea, that has sunk in!! Thank you blokhead for the matrix you took the time to create for me. I've been going over it, trying to get the premise to sink in, and it will. Somehow.. with enough beer maybe... "I guess if perl wants to reserve the right to do something else, you'll have to respect that." Crap, I already have a wife with those processing parameters!

    Thanks for the link chargrill. It's an awesome reference to someone else tormenting themselves, and I am glad no one got mad at each other in my node.

    Thank you everyone, including ysth & chromatic for your time spent on the issue. I appreciate your input and patience in doling (<-sp?) out your insights and experience. I'm going to go dwell over blokhead's matrix and see if I can make sense of my example, AND DO NOT FEAR, I am not planning on implementing this idea anywhere or taking it to heart that this is how Perl treats these parameters. I realize my results (On multiple paltforms & versions) are random acts of consistency! I really am quite sane, in a more or less, sane-ish way!! brusimm muttering to himself - "random acts of consistency", man where did that come from? as he leaves the room.

      Just for grins, I was trying to see what my local perl would produce. To my surprise, my answer was different! But then I tried a few more things and realized I had misstyped the original problem.

      But since I found this interesting, I thought I'd share. (Note, this behavior might also show up in the off-site link from the original poster in my earlier post in this node, but that's old and off-site.)

      $ perl -e 'my $m = 20; print $m++ + $m++, "\n"' 41 $ perl -e 'my $m = 20; print $m++ + ++$m, "\n"' 42 $ perl -e 'my $m = 20; print ++$m + $m++, "\n"' 43 $ perl -e 'my $m = 20; print ++$m + ++$m, "\n"' 44

      s**lil*; $*=join'',sort split q**; s;.*;grr; &&s+(.(.)).+$2$1+; $; = qq-$_-;s,.*,ahc,;$,.=chop for split q,,,reverse;print for($,,$;,$*,$/)

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://593014]
Approved by Corion
belg4mit kicks Perl in the shins, "Unmatched ) in regex; marked by" my ass
[belg4mit]: Nothing wrong with perl -le 'print ":)" =~ /(?^u:^(\:|\:\))$) /'
[Lotus1]: belg4mit are you on a Linix server?
[belg4mit]: yes. the regular expression is generated in code from a text file. that s the regular expression that Perl blows up when it reaches it, but if I use it from the comman line it's fine

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (8)
As of 2017-04-25 16:40 GMT
Find Nodes?
    Voting Booth?
    I'm a fool:

    Results (459 votes). Check out past polls.