Autovivification sucking the life out of me

[shenme]

I came across a strange side effect while using File::Spec::tmpdir. $ENV{TMPDIR} is being created and set to undef, as though it is being autovivified. I pulled the pieces out of File::Spec::Win32 and ::Unix and got it down to this example:

#!perl -w

use strict;
use warnings;

sub see_args { 
    printf "I see args '%s'\n", join("', '",@_); 
    return; 
}

    warn sprintf "In test, at start (1): TMPDIR '%s'\n",
        ! exists $ENV{TMPDIR} ?  '<absent>' : ! defined $ENV{TMPDIR} ?
+ '<undef>' : $ENV{TMPDIR};

    see_args( @ENV{qw(TMPDIR TEMP TMP)} );

    warn sprintf "In test, at end (2): TMPDIR '%s'\n",
        ! exists $ENV{TMPDIR} ?  '<absent>' : ! defined $ENV{TMPDIR} ?
+ '<undef>' : $ENV{TMPDIR};

produces output:

In test, at start (1): TMPDIR '<absent>'
Use of uninitialized value in join or string at test3.pl line 7.
I see args '', 'C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp', 'C:\DOCUME~1\ADMI
+NI~1\LOCALS~1\Temp'
In test, at end (2): TMPDIR '<undef>'

Just to be sure it wasn't the _use_ of the subroutine arguments, I commented out the printf, and got output

In test, at start (1): TMPDIR '<absent>'
In test, at end (2): TMPDIR '<undef>'

so it is the call itself doing the nasty thing to my mind. Can someone make this make sense to me?

[Skeeve]

Your autovivification happens here:

see_args( @ENV{qw(TMPDIR TEMP TMP)} );

If you explicitly tell perl to vivify $ENV{TMPDIR} it will do so. It's not the call doing it, but the parameters you pass in your call.

[Perl Mouse]

Whenever a non-existing hash or array element is used in anything but a simple context, it's autovivificated. The *default* is to autovivificate - it's just that the exceptions happen to be the most often used cases, so it's a surprise when perl autovivifies.

#!/usr/bin/perl

use strict;
use warnings;

my %hash;

sub X {}
my $x;

$x = exists $hash{key1};
printf "key1 %sautovivified\n",  exists $hash{key1} ? "" : "not ";

$x = $hash{key2};
printf "key2 %sautovivified\n",  exists $hash{key2} ? "" : "not ";

$x = exists $hash{key4}{key3};
printf "key3 %sautovivified\n",  exists $hash{key4}{key3} ? "" : "not 
+";
printf "key4 %sautovivified\n",  exists $hash{key4} ? "" : "not ";

($x) = @hash{key5};
printf "key5 %sautovivified\n",  exists $hash{key5} ? "" : "not ";

($x) = @hash{"key6", "key7"};
printf "key6 %sautovivified\n",  exists $hash{key6} ? "" : "not ";
printf "key7 %sautovivified\n",  exists $hash{key7} ? "" : "not ";

X($hash{key8});
printf "key8 %sautovivified\n",  exists $hash{key8} ? "" : "not ";

X(@hash{key9});
printf "key9 %sautovivified\n",  exists $hash{key9} ? "" : "not ";

X(@hash{"keyA","keyB"});
printf "keyA %sautovivified\n",  exists $hash{keyA} ? "" : "not ";
printf "keyB %sautovivified\n",  exists $hash{keyB} ? "" : "not ";  
__END__
key1 not autovivified
key2 not autovivified
key3 not autovivified
key4 autovivified
key5 not autovivified
key6 not autovivified
key7 not autovivified
key8 not autovivified
key9 autovivified
keyA autovivified
keyB autovivified

[shenme]

Wow, thanks for the examples. It would seem "simple usage" is not what people would assume it is. I will go file a bug report against PathTools, but it may be instructive to see how easy it is to fall into this trap. Here is the line of code in File::Spec::Win32.pm that caused me all the grief:

  $tmpdir = $_[0]->_tmpdir( @ENV{qw(TMPDIR TEMP TMP)},
                            'SYS:/temp',
                            'C:\system\temp',
                            'C:/temp',
                            '/tmp',
                            '/'  );

Sure looks innocent, huh? Of course the 'fix' will be to change the references to 'simple' ones,

  $tmpdir = $_[0]->_tmpdir( $ENV{TMPDIR},
                            $ENV{TEMP},
                            $ENV{TMP},

And in answer to Skeeve, yes, I am a "bug-magnet"

[shenme]

But I am not explicitly telling Perl to autovivify, rather I am (intending to) just reference the values, in this case using a slice.

Refer to my simpler example above. Using see_args( @foo{qw(BAR BAZ)} ); causes result:

at start: BAR '<absent>'
at end: BAR '<undef>'

But changing that to see_args( $foo{BAR}, $foo{BAZ} ); results in:

at start: BAR '<absent>'
at end: BAR '<absent>'

This is most perplexing to me.

[Skeeve]

Tell you what: It's perplexing to me the other way around!

Until 2 days ago I thought something like
if ($foo{'BAR'}) ...
would already vivify that hash entry.

But then I learned in the Chatter Box that it doesn't in perl 5.8 (or maybe a bit earlier).

And now I did a test. Compare the output

use Data::Dumper;
%foo=(BAZ=>1);
print Dumper \%foo if $foo{BAR} || 1;
print Dumper \%foo if @foo{BAR,BAZ} || 1;

Output:

$VAR1 = {
          'BAZ' => 1
        };
$VAR1 = {
          'BAZ' => 1
        };



use Data::Dumper;
%foo=(BAZ=>1);
print Dumper \%foo if call($foo{BAR});
print Dumper \%foo if call(@foo{BAR,BAZ});
sub call { return 1 }

Output:

$VAR1 = {
          'BAZ' => 1
        };
$VAR1 = {
          'BAR' => undef,
          'BAZ' => 1
        };


So it's really the fact that the slice is used in a subroutine call. When you replace call with print, autovivification does not happen

So now I ask myself: Is there a bug and you are certified bugfinder!? ;-)

[pg]

This really has nothing to do with autovivification, you never attempted to define $ENV{TMPDIR}, and $ENV{TMPDIR} never existed (through either you or autovivification, unless it was really defined as an environment variable before you enter the program) before or after.

The first argument you passed in is undef, which failed the join. Not surprisingly, comment out printf obviously will get rid of the error. Here is part of your code without that function:

use strict;
use warnings;
use Data::Dumper;

    warn sprintf "In test, at start (1): TMPDIR '%s'\n",
        ! exists $ENV{TMPDIR} ?  '<absent>' : ! defined $ENV{TMPDIR} ?
+ '<undef>' : $ENV{TMPDIR};
print Dumper(@ENV{qw(TMPDIR TEMP TMP)});

Which prints:

$VAR1 = undef;
$VAR2 = 'C:\\DOCUME~1\\someone\\LOCALS~1\\Temp';
$VAR3 = 'C:\\DOCUME~1\\someone\\LOCALS~1\\Temp';

[shenme]

I think you missed the point of the post, which is "how is the variable $ENV{TMPDIR} being autovivified?" I am not caring that the value passed into the subroutine is undefined, since the environment variable is indeed not defined. But I do care that the variable outside the subroutine changes from '<absent>' to '<undef>'.

[pg]

Update: My apologies! Forget what I have said, it was my observation was off, not yours. But it might be a better idea to demo your issue by using Data::Dumper.

use Data::Dumper;

use strict;
use warnings;
my  %foo = ( BAZ => 1 );
sub see_args { };

print Dumper(\%foo);
see_args( @foo{qw(BAR BAZ)} );
print Dumper(\%foo);

It prints:

$VAR1 = {
          'BAZ' => 1
        };
$VAR1 = {
          'BAZ' => 1,
          'BAR' => undef
        };

Yes, autovivification is in working.

Your observation was off. In your original post, you were confused by the fact that the $ENV{TMPDIR} "appears" to have a value as you saw it through that sprintf. No, you didn't see the value of $ENV{TMPDIR}. Your sprintf printed a constant "<absent>", not the value of $ENV{TMPDIR}.</strick>