About "state" variables in Perl 5.10

[citromatik]

Hi all

"state variables" are a new feature in Perl 5.10 that "are similar to my variables, but are declared with the state keyword in place of my. They're visible only in their lexical scope, but their value is persistent: unlike my variables, they're not undefined at scope entry, but retain their previous value."

So, as I understand this, this new kind of variables is like "static" variables in C and allow you to write code like:

sub counter {
  state $count = 0;
  $count++;
}

Before Perl 5.10 you can do that with:

{
  my $count = 0;
  sub counter {
    $count++;
  }
}

This way allows you to share a common $count variable between more than one sub:

{
  my $count = 0;
  sub counter1 {
    $count++;
  }
  sub counter2 {
    $count += 2;
  }
}

Since I find the latter very simple to follow and understand I would like to know if there are differences between them and when one or the other should be used

Thanks in advance!

citromatik

[TimToady]

There is some overlap in applicability, but the main semantic difference between

    {
        my $x = 42;
        my $s = sub { $x++ };
        say &$s();
    }

and

    my $s = sub { state $x = 42; $x++ };
    say &$s();

is that the former initializes $x before &$s is created, while the latter does not initialize $x until the first time you call &$s. Note also that with state each created anonymous sub gets its own copy of $x, since technically it's a different lexical scope (or a different pad for the same lexical scope, depending on how you look at it).

This makes little difference with the two snippets above, since in the first example the my also creates a new $x every time you execute the snippet, but if you created $x only once, and then created the closure multiple times in a loop, you'd notice that the single my $x is shared among them.

However, that difference is not why we introduced state variables. The big win is the psychological one of not having to look outside of the sub to find the definition of $x. (That, and we wanted it as a primitive in Perl 6 so that people could write stateful macro constructs without forcing the user to define the state variable externally; in fact, we also use it ourselves in order to implement the stateful flipflop operator without having to build the flipflop in as a primitive, as it is in Perl 5.)

[citromatik]

Note also that with state each created anonymous sub gets its own copy of $x

Hmmm... but you can get the same result with the closure approach:

sub get_counter {
   my $x = 42;
   return sub {
      $x++;
   }
}
my $s = get_counter();
say &$s();

However, that difference is not why we introduced state variables. The big win is the psychological one of not having to look outside of the sub to find the definition of $x.

Nice, that reason convince me (and the one that kyle gave below).

Thanks

citromatik

[kyle]

The main difference I see is that the initialization of state variables only happens when the function is called. Consider:

use feature qw( :5.10 );

sub uncalled_sub {
    state $huge = huge_data_structure();
    return;
}

print 'mem usage: ', my_mem(), "\n";

sub huge_data_structure {
    return [ ( 'x' x 1_000 ) x 100_000 ];
}
sub my_mem {
    my ($proc_info)
        = grep { $_->[2] == $$ } map { [ split ] } `ps l | tail -n +2`
+;
    return $proc_info->[6];
}
__END__
mem usage: 11220

Compare to:

{
    my $huge = huge_data_structure();
    sub uncalled_sub {
#        state $huge = huge_data_structure();
        return;
    }
}

print 'mem usage: ', my_mem(), "\n";
__END__
mem usage: 116132

Both tests were performed with Perl 5.10.0. You get the same kind of side effect (with time instead of memory) if initialization is a long_arduous_task().

Consider also:

print counter(), "\n";
{   
    my $count = 0;
    sub counter { $count++ }
}
print counter(), "\n";
__END__
0
0

Compare to:

use feature qw( :5.10 );

print counter(), "\n";
sub counter {
    state $count = 0;
    return $count++;
}
print counter(), "\n";
__END__
0
1

Under ordinary circumstances, you won't notice either problem. Your initialization is fast, and it happens inside something you used, not in the middle of your program.

I think it also helps maintainability. You have the variable that's relevant to the function inside it instead of outside it. Even if they're right next to each other (as you have them), they could get separated later, and you don't want one of them left behind in some refactoring.

[citromatik]

The main difference I see is that the initialization of state variables only happens when the function is called

Yes, that is a good difference. Thanks!

Consider also:

print counter(), "\n";
{   
    my $count = 0;
    sub counter { $count++ }
}
print counter(), "\n";
__END__
0
0

This one is easy to overcome, just putting the code in a BEGIN block:

print counter(), "\n";
BEGIN{   
    my $count = 0;
    sub counter { $count++ }
}
print counter(), "\n";
__END__
1
0

citromatik

[Limbic~Region]

citromatik,
In addition to what others have said, in the simple case (single sub), there isn't the need for the extra scope. When you need more than one sub to share, the old trick still works for that. There are many more things you can do with state though. See How will you use state declared variables in Perl6? for a more in-depth discussion. I used the state variables to create pseudo objects before Pugs had objects. You can see a perl 5.10.0 example here.

[shmem]

Since I find the latter very simple to follow and understand I would like to know if there are differences between them

Er... between what?

Do you know the difference between a hippo?

<Reveal this spoiler or all in this thread>

Each sub has its own scope, so sharing a lexical state variable via state $var is not an option, you have to allocate that var in an outer scope. my and state make no difference there.

[exussum0]

Neat. C style static variables. I do have one curious thought. Didn't oop solve this via objects/classes? Was there a great demand for it? What inspired this neat little feature?

[grinder]

One particular itch that needed scratching, apart from some of the points that TimToady makes, is that it got rid of the ugly state-with-a-my hack that went something like:

  my $persist if 0;

That particular construct breaks through the abstraction of the language and hits an oddity in the way lexicals are implemented. The result of which was, when used in a routine, a variable that maintained its previous value next time the routine was called.

It was a sufficiently desirable trick that it was independently discovered time and again, by people who understood the language sufficiently deeply, and tried it out and were happy to discover that it worked.

Unfortunately, it was also regularly discovered by people who had no idea what was going on, and it led to bugs that were difficult to understand and fix (especially when the if 0 was buried under a complex expression).

So one of things that went into 5.10 was the outlawing of my $foo if 0 to stop people shooting themselves in the foot, and the introduction of state variables, for those who needed them.

[tye]

So one of things that went into 5.10 was the outlawing of my $foo if 0

Why still allow my $foo if $bar; ? Such seems, if anything, worse than the if 0 construct. I can understand putting it through a deprecation cycle but not even that was started.