Should calling 'exists' create a hash key?

[mje]

#!/usr/local/bin/perl -w
use strict;
use Data::Dumper;
my $x;
print "fred defined\n" if (exists($x->{fred}));
print "defined\n" if (exists($x->{fred}->{dave}));
print Dumper($x);
print "fred defined\n" if (exists($x->{fred}));

Should calling exists spring $x->{fred} into existance?

2005-09-20 Retitled by g0n, as per Monastery guidelines
Original title: 'Should this happen?'

[Fletch]

Not the first case, but calling exists( $x->{fred}->{dave} ) autovivifies a hashref in $x->{fred}.

[reasonablekeith]

You fallen into a common* gotcha, so don't feel too bad. You might want to browse through the following perlarchive article, which explains autovivification in full...

http://www.perlarchive.com/___TLC/7026.shtml

* updated spelling mistake

[pg]

This is called autovivification. To me, there are two types of autovivification, the ones that are useful, and the ones that are harmful (gotcha).

I have always seen the example you gave here as the harmful type, as it goes too far to guess your purpose and 99% of the time, nobody wants that free service.

But sometime it is useful, for example:

use strict;
use warnings;
my $a;
$a->{"a"}{"b"}{"c"} = 1;

In this case, Perl will autovivify $a->{"a"} and $a->{"a"}{"b"}, and you don't need to initialize them explicitly. This is a clever idea, if I live on the six floor of 666 on street 888, obviously both street 888 and No. 666 exist. But it is ridiculous to assume that both No. 666 and street 888 exist, just because someone has asked whether "No.666 on street 888 has a six floor".

Also in this case:

my @a;
a[100] = 2;

Perl will autovivify a[0] through a[99]. a[0] through a[99] come to existance without you explicitly doing so. This is different from languages like c or Java, in which you have to explicitly do it by allocate memory/new.

There are also other types of autovivifications.

[Fletch]

Your second example with an array isn't really autovivification. Perl hashes are sparse (elements aren't stored contiguously in memory), whereas Perl arrays aren't sparse (the sequence of elements is stored contiguously in memory). When you do my %h; $h{ 100 } = 2; the one SV* is stored; when you do your example, Perl has to allocate room to store 101 SV*s and store the SV* for the 2 in the 101st (the other 100 contain pointers to sv_undef).

Autovivification is when Perl allocates a reference to a new anonymous hash or anonymous array on the fly; this is just how arrays work. Now if you did $a[100][42] = 3.14159 that would be autovivification in that it would create a new arrayref (again, with room for 43 elements preallocated) and store it into $a[100] and then store 3.14159 into the last element.

The difference from C or Java is that Perl will automatically extend arrays for you (which again isn't autovivification), whereas the other languages you do have to manually allocate or extend them.

[pg]

In the case of $a[100][42] = pi, obviously there is autovivification. That has no difference from $a->{"a"}{"a"} = pi.

Whether $a[100] makes $a[0] through $a[99] autovivified is not a black white thing, and I am not the first person who classifies that as autovivification. When it is valid to look at this from a perl internal point of view as you did, it is at least not less valid to look at this from a logical point of view. Look at this:

use strict;
use warnings;

my @as;
$as[10] = 0;
for my $a (@as) {
    print ".";
}

this will print 11 dots, not 1 or 12.

To further pursuit a single correct answer to this question is not very useful, as autovivification is not a clearly defined concept, but rather a commonly used not-clearly-defined terminology that is often expressed through examples.

[tye]

Avoiding this is one use of Data::Diver.

[svenXY]

Hi,
yes, I have had a similar problem.

exists($x->{fred}->{dave})

implicitely (automagically???) creates $x->{fred}
Regards,
svenXY

[mje]

Thanks for the responses. Had a realised this was autovivification I would have found the answer. I see the point of autovivification when you want to create deep structures and the time/code it saves you but am slightly surprised exists is not a special case.

[graff]

As I'm sure you have figured out already, if you want to avoid autovivication in this sort of operation, you can either use Data::Diver (as suggested by tye), or you can rearrange your conditionals:

if ( exists( $$x{fred} )) {
   print "fred defined\n";
   print "dave defined\n" if ( exists( $$x{fred}{dave} ));
}

Obviously, if you need to do this sort of thing a lot, with arbitrarily deep structures, using the module will make your coding a lot easier and cleaner.

[Aristotle]

The minimal form is actually:

my $x;
$x->{foo};
print $x; # it's a hash!

and likewise

my $x;
$x->[0];
print $x; # it's an array!

In other words, Perl automatically turns any undefined lvalue into an an anonymous hash or array whenever you treat it like one. This is useful because it lets you do things like grouping input on multiple keys easily:

my $event;

while( <> ) {
    chomp;
    my ( $time, $lane, $factory, $event ) = split " ", $_;
    $event->{ $factory }{ $lane }{ $time } = $event;
}

If Perl didn’t autovivify undefs, you’d have to write something like

my $event = {};

while( <> ) {
    chomp;
    my ( $time, $lane, $factory, $event ) = split " ", $_;

    $event->{ $factory } = {}
        if not exists $event->{ $factory };

    $event->{ $factory }{ $lane } = {}
        if not exists $event->{ $factory }{ $lane };

    $event->{ $factory }{ $lane }{ $time } = $event;
}

Depending on how deep the structure is and whether any branches are optional depending on what conditions, you’d might end up spending a whole lot of time fiddling around to put the right things in the right places in the right succession. With autovivification, Perl just DWIMs.

But as you’ve seen, sometimes this is more than you were asking for. Because exists is only a function, not an operator, the lookup happens independently, and by the time exists sees the result, the deed has already happened.

You need to keep this in mind whenever you write multi-level lookups.

[snoopy]

A couple of possible code hacks to avoid autovivication in:

print "defined\n" if (exists($x->{fred}->{dave}));

1. Use last to abort the statement (adapted from Re: Shortcut operator for $a->{'b'}=$b if $b;):

{ print "defined\n" if (exists(%{$x->{fred}||last}->{dave})); }

2. Redirect to an anonymous empty hash ref:

print "defined\n" if (exists(%{$x->{fred}||{}}->{dave});

[tomazos]

Autovivify in this usage is a term coined to mean that perl automatically creates a variable the first time you talk about it. Vivify means to "bring to life" (or worse "to operate on something while it is alive"). Tack on "auto" and you get the general idea.

If you do not assign to a newly autovivified variable it will default to the undefined value (undef).

use strict and use warnings help.

The difference between (a) a hash key that exists and is defined, (b) a hash key that exists and is not defined, and (c) a hash key that does not exist - is a common point of confusion.

-Andrew.