Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things

DWIM: autoloading classes

by Ovid (Cardinal)
on Sep 17, 2003 at 00:35 UTC ( #292009=perlmeditation: print w/replies, xml ) Need Help??

Update: How bizarre! I haven't been on Perlmonks all day, but I come here to make this weird post only to discover (after I post) a freakishly similar idea discussed today.

An interesting talk with a coworker about Smalltalk has inspired me to think about a new, rather curious module.

Often, when working on a project, I have many related packages. It can get awfully annoying to find this at the top of all of my code:

use Foo::Dates; use Foo::Customer; use Foo::Order; use Foo::Company; use Foo::Company::SalesReps; use Foo::Extremely::Long::Package::Name;

In Smalltalk, you just use (as in "make use of", not to be confused with Perl's keyword "use") the classes. In Java, there is at least a useful import keyword:

import Foo.*;

I started to wonder why I can't do that in Perl. After talking to him about implementations, I took a snippet he wrote and came up with something like the following code:

sub HTML::TokeParser::Simple { my $module = (caller(0))[3]; undef *{$module}; eval "require $module"; if ($@) { require Carp; Carp::croak "No such module: $module"; } return $module; } package main; my $parser = HTML::TokeParser::Simple->new(\*DATA); __DATA__ <html> <head> ... more html

If you don't understand that code, what it does is create a function in the "HTML::TokeParser" namespace that undefs itself (so it won't be called more than once), uses the class in question and then returns the fully qualified name as a string.

So what if I were to create a module that did something like this:

use Class::WhenNeeded 'Foo'; my $customer = Foo::Customer->new($customer_id); my $order = Foo::Order->new;

In other words, we can skip the long lists of "use" and require declarations. I specifically mention "Class" in the package name as I expect that function oriented modules would be more difficult to manage due to exporting issues.

Another interface -- but more dangerous since it could have namespace collisions if you use subs with upper-case first letters:

use Class::WhenNeeded Foo => 'no_top_level'; my $customer = Customer->new($customer_id); my $order = Order->new;

Can anyone think of strong reasons why this would or would not be a good idea? I can see some problems with this, but it's an interesting idea.


  • Compilation problems that show up at compile time would now be delayed until runtime.
  • Exporting would likely break, (but many OO modules don't export anything).
  • A recursive search through @INC from the namespace specified could generate a lot of subroutines and slow compilation.


New address of my CGI Course.

Replies are listed 'Best First'.
Re: DWIM: autoloading classes
by liz (Monsignor) on Sep 17, 2003 at 09:14 UTC
    Paranoid as I am, I am wondering whether the automatic loading of Foo::* modules isn't opening some serious security holes, because you can never know whether someone hasn't put a "Foo::Takeovereverything" module somewhere in @INC (especially with "." usually being part of @INC). Of course, this could be considered a feature as well.

    Personally, I like the following idiom better for object oriented modules, as it makes it easier to use a lot of modules _and_ still keep a better overview:

    use Foo qw(Bar Baz);
    which would be equivalent to:
    use Foo (); use Foo::Bar (); use Foo::Baz ();

    Foo::import would look something like this:

    my @module = qw(Bar Baz); # allowable modules my %module = map {$_ => 1} @module; # easy lookup of modules sub import { my $class = shift; return if $class ne 'Foo'; # allow import to be inherited by submo +dules my @require; # list of modules to be required if (@_) { foreach (@_) { if ($_ eq ':all') { @require = @module; } elsif ($module{$_}) { push @require,$_; } else { warn "Don't know how to load Foo::$_\n"; } } } else { @require = @module; } foreach (@require) { next if eval "require Foo::$_;"; # string eval for easiness warn "Foo::$_: $@"; # could be die also if you prefer } } #import

    With apologies to those for whom this would be bleedingly obvious.


      While I'm too tired to implement it, one should be able to add import functionality to the above code so that symbols exported by Foo::Bar via "use Foo 'Bar';" are exported to the calling namespace. That should be doable by testing whether Foo::Bar uses Exporter to handle import() and friends and use the export_to_level() routine from Exporter. If the module uses its own import routine, one might... well... do evil things with eval and package.
        Not that I want to export anything: I do everything OO and nothing gets exported in my modules. If there are any global flags that would need to be set, I use class method accessors for them.

        But anyway, but I fail to see what more evil things you could do with an exporter that you couldn't already do.

        So what ev(i|a)l things exactly are you referring to?


Re: DWIM: autoloading classes
by Abigail-II (Bishop) on Sep 17, 2003 at 01:29 UTC
    If you are not exporting, why not create a file
    package Foo; use Foo::Dates; use Foo::Customer; use Foo::Order; use Foo::Company; use Foo::Company::SalesReps; use Foo::Extremely::Long::Package::Name;

    and use use Foo.

    And if you want to take care of exporting, you could do something like (untested):

    package Foo; use Foo::Dates (); use Foo::Customer (); use Foo::Order (); use Foo::Company (); use Foo::Company::SalesReps (); use Foo::Extremely::Long::Package::Name (); sub import { Foo::Dates -> export_to_level (2, @_); Foo::Customer -> export_to_level (2, @_); Foo::Order -> export_to_level (2, @_); ... }


      I can see from my original node that I (once again) was not terribly clear. The intent of this idea was to ensure that I had a general purpose module that I could point to any namespace and it would recursively travel through the directories (via File::Find or File::Find::Rule) and prepare those subroutines as I outlined above. Thus, I wouldn't have to change code in any of the classes found.

      Further, by just adopting the syntax you listed in Re: The costs of packages, I think I might get exporting to work, also. What follows is a mish-mash of pseudo-code and Perl:

      package Class::WhenNeeded; use File::Find; sub import { my ($class, $top_package, $no_top_level) = @_; my @dirs = map { find_top_level_dir($_, $top_package) } @INC; foreach my $dir (@dirs) { find({ wanted => \&create_package_sub($no_top_level), preprocess => \&perl_packages, }, $dir); } } sub create_package_sub { my $strip_top_level = shift; my $module = make_module_name($File::Find::name, $strip_top_ +level); my $call_pack = (caller(1))[1]; if (defined &{$module}) { require Carp; Carp::croak "Function $module already defined"; } *{$module} = sub { undef *{$module}; eval <<" END_EVAL"; package $call_pack; use $module; # export goes to calling package? END_EVAL if ($@) { require Carp; Carp::croak "Could not load module: $module"; } return $module; } }

      Hopefully that clears things up.


      New address of my CGI Course.

        Your aiming at a somewhat different goal to me, in that my package don't exist as files and only come into being when they are used, but I really like where your going too.

        I'm not sure if it fits with your intent, but I really like the idea of

        use WhenNeeded qw[ HTTP::* HTML::* ]; my $headers = new HTTP::Headers; ... my $request = new HTTP::Request; ... my $content = decode( $content );

        But maybe that syntax is just too Javan :)

        Examine what is said, not who speaks.
        "Efficiency is intelligent laziness." -David Dunham
        "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller
        If I understand your problem, I can solve it! Of course, the same can be said for you.

•Re: DWIM: autoloading classes
by merlyn (Sage) on Sep 17, 2003 at 02:02 UTC

      Actually, I didn't know about that module, but what I was looking at is still different. Even the Class::Autouse is considerably different:

      use Class::Autouse qw{CGI Data::Manip This::That};

      Mine finds all classes below a top level namespace. For example, if you have Foo::Bar, Foo::Bar::One::More, Foo::Bar::Two::Bad and Foo::Bar::Three::SaCharm, you could do make all of them "autoused" with this:

      use Class::WhenNeeded 'Foo::Bar'; my $bad_luck = Foo::Bar::Two::Bad->new; # or possibly use Class::WhenNeeded 'Foo::Bar' => 'no_top_level'; my $good_luck = Three::SaCharm->new;

      Perhaps my response to Abigail-II will clear it up. (Or perhaps it won't :)


      New address of my CGI Course.

        It looks like Class::Autouse has a "superloader" option that will load any class as soon as you call a method on it.
Re: DWIM: autoloading classes
by tsee (Curate) on Sep 17, 2003 at 09:35 UTC
    Does this weird piece of code do what you want? (Needless note to others: Don't use this code. Note to self: Don't write such code!)
    package StupidAutoloader; use strict; use warnings; use Carp; our %Classes; sub import { my $class = shift; my @hierarchies = @_; %Classes = map {($_, undef)} @hierarchies; *UNIVERSAL::new = sub { my $proto = $_[0]; my $class = ref($proto) || $proto; croak "Could not find method new() in package '$class'." if ref($proto) or exists $INC{$class} or not in_hierarchy(\%Classes, $class); eval "use $class;"; croak "Error loading module '$class': $@" if $@; no strict 'refs'; goto &{"${class}::new"}; }; } sub in_hierarchy { my $h = shift; my $c = shift; return 1 if exists $h->{$c}; my @levels = split /::/, $c; my $accum; foreach (@levels) { $accum .= $_; return 1 if exists $h->{"${accum}::*"} } return 0; } 1;
    #!/usr/bin/perl use strict; use warnings; use StupidAutoloader 'Data::*'; my $d = Data::Dumper->new([]); print "$d\n"; my $g = Data::Grouper->new(a => []); print "$g\n";

      That's a very interesting piece of code. It never ceases to amaze what sort of things we can do with Perl. The only problem is that not all constructors are named 'new' :)


      New address of my CGI Course.

        That problem is alleviated by fiendishly abusing AUTOLOAD in UNIVERSAL. (There are people who'd kill for this... if you use it in production!) ---
        #!/usr/bin/perl use strict; use warnings; use StupidAutoloader qw/Math::* Data::*/; my $function = Math::Symbolic->parse_from_string('1/2*m*v^2'); print $function->simplify(), "\n"; # Doesn't work because Math::Symbolic::Operator *implicitly requires o +ther # modules in the Math::Symbolic:: hierarchy: # my $op = Math::Symbolic::Operator->new(...); my $cmplx = Math::Complex->new(3, 2); print $cmplx; print Data::Dumper->Dump([$function]); # Even this works: use StupidAutoloader '*'; my $cgi = CGI->new(); print $cgi->header();
        package StupidAutoloader; use strict; use warnings; use Carp; our %Classes; our $AUTOLOAD; *UNIVERSAL::AUTOLOAD = sub { my $proto = $_[0]; my $class = ref($proto) || $proto; my $method_name = $AUTOLOAD; die "Could not determine method name." if not defined $method_name; $method_name =~ /::(\w+)$/ or die; $method_name = $1; croak "Could not find method new() in package '$class'." if ref($proto) or exists $INC{$class} or (not _in_hierarchy(\%Classes, $class) and not exists $Classes +{'*'}); eval "use $class;"; croak "Error loading module '$class': $@" if $@; no strict 'refs'; goto &{"${class}::${method_name}"}; }; sub import { my $class = shift; my @hierarchies = @_; foreach (@hierarchies) { $Classes{$_} = undef; } } sub unimport { my $class = shift; my @hierarchies = @_; if (not @hierarchies) { %Classes = (); } foreach (@hierarchies) { delete $Classes{$_}; } } sub _in_hierarchy { my $h = shift; my $c = shift; return 1 if exists $h->{$c}; my @levels = split /::/, $c; my $accum; foreach (@levels) { $accum .= $_; return 1 if exists $h->{"${accum}::*"} } return 0; } 1;
Re: DWIM: autoloading classes
by adrianh (Chancellor) on Sep 17, 2003 at 07:30 UTC

    I have to admit if I was finding:

    use Foo::Dates; use Foo::Customer; use Foo::Order; use Foo::Company; use Foo::Company::SalesReps; use Foo::Extremely::Long::Package::Name;

    at the top of all my files I would be saying to myself "Gosh. We seem to have a lot of very tight coupling and duplicate code here. Probably need to do some refactoring" :-)

Re: DWIM: autoloading classes
by IlyaM (Parson) on Sep 17, 2003 at 09:34 UTC
    I haven't tried it but there is CPAN module import which autoloads related packages (and also aliases them to short names).

    Ilya Martynov,
    CTO IPonWEB (UK) Ltd
    Quality Perl Programming and Unix Support UK managed @ offshore prices -
    Personal website -

Re: DWIM: autoloading classes
by idsfa (Vicar) on Sep 17, 2003 at 15:57 UTC
    This subject seems to be a recurring theme. The snippet referred to allows more complex selection mechanisms than just *.
Re: DWIM: autoloading classes
by pdcawley (Hermit) on Sep 19, 2003 at 06:07 UTC
    The problem with deferring loading 'til runtime is that modules that make use of, say CHECK or INIT blocks will fail to work. Admittedly, such modules are usually 'language warping' type tools, but you might be attempting to use them indirectly.

    A better option might be to have an Application class that loads everything the app needs in one fell swoop, which also helps with module codependencies. For the purposes of testing, where you want to make sure that modules you believe to be isolated actually are isolated, you could write yourself an export routine, so you could do:

    use Application qw/Foo::Bar Foo::Baz Foo::Wibble/; # Testcode goes here
    For evilness points you could have Application detect when it's being run via testing code and have it add any 'new' modules to its list of modules to load when used without a module list. However, I think throwing an exception in all cases would be a rather more sensible option.

    Going down that track a little further, if we assume that the only time Application will be used with a module list is during testing (if you want to vary the list of modules for different scripts, subclass Application) then you could have it generate a list of tested modules, then have a 99sanitycheck.t that verifies that Application has been used to explicitly load every module that it will load by default.

    Hmm... I think I might have to find the tuits to write this.

      Agreed, run-time loading is a less than ideal solution for any number of modules (it fubars exports, in addition to the already mentioned CHECK and INIT issues). The whole reason that I was even thinking along those lines was because I was looking to avoid having to rewrite code in the main portion of a program every time a new submodule was added to extend the functionality.

      This trick is really intended for loading all modules which fullfil a predetermined API, even if they were written long after the application which will use them. Strangely enough, I am actually using perl to write reports (in this case on instances of custom applications) and wanted to stop having to rewrite the damned thing every time we came out with a new product. Now I only have to write code which can gather the desired information from each new app and massage it into the common format.

      I'd love to see some way to do this without losing the exports (Win32::TieRegistry's $Registry in particular). Dynamically loading OS-dependant modules would also be nice. Also a million dollars and a pony.

      Remember, when you stare long into the abyss, you could have been home eating ice cream.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlmeditation [id://292009]
Approved by Enlil
Front-paged by Enlil
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others meditating upon the Monastery: (3)
As of 2022-05-18 07:34 GMT
Find Nodes?
    Voting Booth?
    Do you prefer to work remotely?

    Results (68 votes). Check out past polls.