Writing a Perl module that dynamically loads other modules

[mgad]

Hi everyone,

I've been looking through CPAN trying to find some examples of a Perl module that does this, but I haven't come across one that seems like a fit for my specific application. Basically, I'm writing a module that will read a file, slurp in it's contents, parse through the contents until it finds a FILETYPE, load a module based on the FILETYPE, and then continue processing the rest of the file. The end goal is to return a data structure ($self) that contains the important parts of the file (important parts are identified by regexs).

The files I'm reading look something like this:

...A couple of header lines...

FILETYPE=XXXXXX

...The useful information I'm after...

The module I've created thus far looks like this:

package My::Package;
use strict;
use warnings;

...other module stuff...

sub read_slurp {

   # "worst-case-scenario" way of splitting up the 
   # file into its separate lines

   foreach ( split /(?:\015{1,2}\012|\015|\012)/, shift ) {

      # Skip comments
      if ( /^(?:\#|\!|$)/ ) {
         next;
      }

      ######################################
      # Parse the Filetype info
      ######################################
      if ( /^FILETYPE=(.*)\s*$/ ) {
         $self->{'FILETYPE'} = "$1";
         next;
      }

      ... LOAD THE APPROPRIATE MODULE ...
      ...     AND CONTINUE PARSING    ...

The individual modules to be loaded would be lists of if/then regexs that would identify info and save it to the data structure like so:

      ######################################
      # Parse the blockname
      ######################################
      if ( /^blockname\s+(.*)\s*$/ ) {
         $self->{'blockname'} = "$1";
         next;
      }

      ######################################
      # Parse the list info
      ######################################
      if ( /^list\s+(\d+)\s+(.*)\s*$/ ) {
         $self->{'list'}->{"$1"}->{"$3"} = "$2";
         next;
      }

      ...ETC ETC ETC...

Any suggestions of places to look or other modules that do something similar? I've been looking over the source code of tons of other modules (a useful exercise anyway) but I figured having some external input from experts wouldn't hurt.

Cheers and thanks!

[stephen]

I'm assuming that you have some way of going from your file type to a module name (i.e. if the filetype is "foo" you load the "My::Module::Foo" module). I'm also assuming that you're not too worried about security, or that you're somehow validating these inputs.

You have a couple of choices for how to dynamically load a module based on the name.

Before we get into the technical aspect of how to do this, first we should consider *why* you want to do dynamic loading. If you have a manageable number of file formats, you can just 'use' them in your script (or put them all in the same file) and just call them when you need them. Something like:

# up at the top:
use My::Format::Foo;
use My::Format::Bar;

# Down later
    ######################################
      # Parse the Filetype info
      ######################################
      if ( /^FILETYPE=(.*)\s*$/ ) {
         $self->{'FILETYPE'} = "$1";
         next;
      }


      my $parser_class = 'My::Format::' . ucfirst $self->{'FILETYPE'};
      my $data = $parser_class->parse($_);

One way is to use the Module::Load module from CPAN. That would let you do:

# Up top
use Module::Load;


# Down in the parsing
my $module_name = 'My::Format::' . ucfirst $self->{'FILETYPE'};
load $module_name;
my $data = $module_name->parse($_);

Alternatively, you can use 'require', like so:

my $module_filename = 'My/Format/' . ucfirst($self->{'FILETYPE'}) . '.
+pm';
require $module_filename;
my $module_classname = 'My::Format::' . ucfirst($self->{'FILETYPE'};
$module_classname->import();
my $data = $module_classname->parse($_);

If you can avoid the dynamic loading, I'd recommend doing so, and just 'use' the expected modules up front. That way you'll reduce complexity and see any errors at load-time. Also, it's a good idea to validate the module names somehow-- a typo in a format type will otherwise show up as a missing module error, and that can get confusing. Good luck!

[mgad]

Stephen - Thanks for the info, I'll definitely take a look at Module::Load. To answer your questions:

>>if the filetype is "foo" you load the "My::Module::Foo"

Thats correct, that is exactly what I was planning to do.

>>you're not too worried about security, or that you're somehow validating these inputs

Correct and correct.

>>If you have a manageable number of file formats

That's the problem and exactly why I'm looking at loading the other modules dynamically. I'm already at 30+ filetypes and by the time everything is finished there will be around 75 total, maybe even more. Loading them all "up front" would be a bit of a pain.

[karlgoethebier]

Why "dynamic"? Common sense tells me (OK - it's denglish isn't it?): Just load them all before doing anything and then do that if/else/switch stuff. Or do you have hundreds of file types ;-)

Regards, Karl

[mgad]

Hi Karl - See above, I don't have hundreds of filetypes but I do have more than seventy and (IMHO) that's too many to load up front. :)

[karlgoethebier]

Hi mgad, what kind of task is this (so many file types)? Perhaps you can provide some more information?

Best regards, Karl