AUTOLOAD for variables?

[dpuu]

Is it possible to do the equivalent of an AUTOLOAD, but for accesses to undefined (un-existing) variables, not functions?

The basic scenario is that I'm maintaining some code that uses Perl as the format of a configuration file, and some people's config files access variables without pre-declaring them. There's a performance problem with the overlying script, becasue its taking a long time to calculate/fetch the values of all the variables that could be accessed in this config file, even though only a small number of variables are used in a given config file. We'd like to avoid modifying user-environments, and fix the performance by tweaking the master-script.

The current code that reads a config-file simply does:

  sub read_config_file {
    my ($filename) = @_;
    package ConfigFile;
    do "$filename";
    if (length $@) { ... }
  }

So what I'd really like to do is to trap accesses to non-existing (scalar) variables in package "ConfigFile", and substitute a value that I determine using some expensive function call. E.g.

  tie %ConfigFile::, "ValueFetcher"

Unfortunately, I can't work out how to tie a symbol table (nor even if I really want to)

One option I've considered is to simply tie every known variable that anyone might want to access, and thus defer fetching the actual value until later. But this feels very, um, un-lazy. (And even working out what variables could be accessed can be somewhat expensive).

If there's no better way to do it, then I guess I'll have to bite the bullet and find an intrusive solution; and then get everyone to change their config files. But doing that with hundreds of users is not something I look forward to.

[tachyon]

When you access a variable that does not exist perl will AUTOVIVIFY it (ie create it for you). Although TheDamian was doing some work on an AUTOVIVIFY sub (equivalent to AUTOLOAD but for vars) AFAIK you can't currently catch access to undefined vars.

It sounds like you have some significant design issues. Perl code for config files + lots of users is an *interesting* (scary) concept. What is to stop someone inserting arbitrary code that will then run with the perms of you master script?

Your explanation of the problem you are trying to solve may make sense to you but is far from clear at this end. You have a performance problem and you say it is loading the config files. Are you sure? What is actually in these files? Why do you think delaying loading will speed things up? Is this a CGI? Are you using mod_perl? ?????

[dpuu]

I was trying not to get into too many details -- they aren't terribly important -- except for finding alternative solutions that don't involve the AUTOVIVIFY method.

Yes, the loads of users of perl-config files is kinda scary -- especially when you see how some people take advantage of having perl. The context has nothing to do with web. It's actually the test environment for our hardware (chip) development. People configure the test scripts by grabing configuration from a database of chip-specific information -- and that database has grown somewhat big over the years (along with the number of scripts that access it).

You are right that there are some serious design issues -- the sort that take a fair amount of time to sort out. We basically have to trawl through a deeply tested directory structure, over nfs, to find about 50K files in the leaves -- then parse those files into a flat namespace for the config file (its not a real database -- which may be part of the problem. Its not quite as bad as it sounds, because we can reduce the scanning with some caching) Then a config file uses only a small number of the values we supply. Multiply this onto a farm with multiple thousand CPUs, and soon you're talking real money!

It really is tremendously ugly (and it may be impossible to reverse-engineer a spec). I was hoping to find a quick fix to use as a stop-gap measure while sorting out the real problems. Perhaps if there is no quick fix, then there's a greater incentive to apply the resources needed to clean it up. I guess this is what happens when a startup becomes big, quickly.

[tachyon]

flat namespace for the config file (its not a real database -- which may be part of the problem

It really depends on how it actually works in practice but it sounds like your end data structure could work better. Depending on all sorts of factors an RDBMS or using a flatfile that stores a hash data structure, serialised with say Storable may help. With the Storable suggestion you would parse the data into a hash (very fast access) then store it into a flatfile with Storable. Anything that needs access just loads the file and gets the whole config hash ready to go. The net result is you effectively spend memory to gain speed.

[Chmrr]

It's possible to do, if you hook into the warning that gets tossed when a variable is only used once. For example:

#!/usr/bin/perl -w
use warnings;

BEGIN {
  use Lingua::EN::Words2Nums;
  $SIG{__WARN__} = sub {
    if ($_[0] =~ /Name "(.*)" used only once/) {
      my $var = $1;
      my ($num) = $var =~ /::(\S+)/;
      $num =~ tr/_/ /;
      $$var = words2nums($num);
    } else {
      warn @_;
    }
  };
}

print $five + $seventeen, "\n";

However, this is firmly in the "sick trick" arena, rather than "maintable code." Heed the other monks' advice to look for better ways to go about this.

Update: Oh, yeah -- this obviously only works if you only refer to the variable once in your code. This may be more trouble than it is worth, obviously. It also won't catch things like $foo = "five"; print $$foo;

[dpuu]

I like the trick but, unfortunately, the evil of my users far surpasses it. Its not uncommon to see code such as:

  my $value= ${ uc join "_", @$reg_spec{ qw( device reg field ) } };

You really don't want to see some of the more nasty code. Honest. Especially not the 70000 line regex (its one saving grace is the /x modifier). And perhaps not the script that parses another script's --verbose output. Or perhaps ...

[dragonchild]

Create an API. Create a set of expected standards. Give your users 3-6 months to comply with those standards, then issue the smack-down. Period. You can't do everything for everyone.