Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris
 
PerlMonks  

Writing a Perl module that dynamically loads other modules

by mgad (Novice)
on Mar 01, 2013 at 19:43 UTC ( #1021324=perlquestion: print w/ replies, xml ) Need Help??
mgad has asked for the wisdom of the Perl Monks concerning the following question:

Hi everyone,

I've been looking through CPAN trying to find some examples of a Perl module that does this, but I haven't come across one that seems like a fit for my specific application. Basically, I'm writing a module that will read a file, slurp in it's contents, parse through the contents until it finds a FILETYPE, load a module based on the FILETYPE, and then continue processing the rest of the file. The end goal is to return a data structure ($self) that contains the important parts of the file (important parts are identified by regexs).

The files I'm reading look something like this:

...A couple of header lines... FILETYPE=XXXXXX ...The useful information I'm after...

The module I've created thus far looks like this:

package My::Package; use strict; use warnings; ...other module stuff... sub read_slurp { # "worst-case-scenario" way of splitting up the # file into its separate lines foreach ( split /(?:\015{1,2}\012|\015|\012)/, shift ) { # Skip comments if ( /^(?:\#|\!|$)/ ) { next; } ###################################### # Parse the Filetype info ###################################### if ( /^FILETYPE=(.*)\s*$/ ) { $self->{'FILETYPE'} = "$1"; next; } ... LOAD THE APPROPRIATE MODULE ... ... AND CONTINUE PARSING ...

The individual modules to be loaded would be lists of if/then regexs that would identify info and save it to the data structure like so:

###################################### # Parse the blockname ###################################### if ( /^blockname\s+(.*)\s*$/ ) { $self->{'blockname'} = "$1"; next; } ###################################### # Parse the list info ###################################### if ( /^list\s+(\d+)\s+(.*)\s*$/ ) { $self->{'list'}->{"$1"}->{"$3"} = "$2"; next; } ...ETC ETC ETC...

Any suggestions of places to look or other modules that do something similar? I've been looking over the source code of tons of other modules (a useful exercise anyway) but I figured having some external input from experts wouldn't hurt.

Cheers and thanks!

Comment on Writing a Perl module that dynamically loads other modules
Select or Download Code
Re: Writing a Perl module that dynamically loads other modules
by stephen (Priest) on Mar 01, 2013 at 20:17 UTC

    I'm assuming that you have some way of going from your file type to a module name (i.e. if the filetype is "foo" you load the "My::Module::Foo" module). I'm also assuming that you're not too worried about security, or that you're somehow validating these inputs.

    You have a couple of choices for how to dynamically load a module based on the name.

    Before we get into the technical aspect of how to do this, first we should consider *why* you want to do dynamic loading. If you have a manageable number of file formats, you can just 'use' them in your script (or put them all in the same file) and just call them when you need them. Something like:

    # up at the top: use My::Format::Foo; use My::Format::Bar; # Down later ###################################### # Parse the Filetype info ###################################### if ( /^FILETYPE=(.*)\s*$/ ) { $self->{'FILETYPE'} = "$1"; next; } my $parser_class = 'My::Format::' . ucfirst $self->{'FILETYPE'}; my $data = $parser_class->parse($_);

    One way is to use the Module::Load module from CPAN. That would let you do:

    # Up top use Module::Load; # Down in the parsing my $module_name = 'My::Format::' . ucfirst $self->{'FILETYPE'}; load $module_name; my $data = $module_name->parse($_);

    Alternatively, you can use 'require', like so:

    my $module_filename = 'My/Format/' . ucfirst($self->{'FILETYPE'}) . '. +pm'; require $module_filename; my $module_classname = 'My::Format::' . ucfirst($self->{'FILETYPE'}; $module_classname->import(); my $data = $module_classname->parse($_);

    If you can avoid the dynamic loading, I'd recommend doing so, and just 'use' the expected modules up front. That way you'll reduce complexity and see any errors at load-time. Also, it's a good idea to validate the module names somehow-- a typo in a format type will otherwise show up as a missing module error, and that can get confusing. Good luck!

    stephen

      Stephen - Thanks for the info, I'll definitely take a look at Module::Load. To answer your questions:

      >>if the filetype is "foo" you load the "My::Module::Foo"

      Thats correct, that is exactly what I was planning to do.

      >>you're not too worried about security, or that you're somehow validating these inputs

      Correct and correct.

      >>If you have a manageable number of file formats

      That's the problem and exactly why I'm looking at loading the other modules dynamically. I'm already at 30+ filetypes and by the time everything is finished there will be around 75 total, maybe even more. Loading them all "up front" would be a bit of a pain.

Re: Writing a Perl module that dynamically loads other modules
by karlgoethebier (Curate) on Mar 01, 2013 at 20:29 UTC

    Why "dynamic"? Common sense tells me (OK - it's denglish isn't it?): Just load them all before doing anything and then do that if/else/switch stuff. Or do you have hundreds of file types ;-)

    Regards, Karl

    «The Crux of the Biscuit is the Apostrophe»

      Hi Karl - See above, I don't have hundreds of filetypes but I do have more than seventy and (IMHO) that's too many to load up front. :)

        Hi mgad, what kind of task is this (so many file types)? Perhaps you can provide some more information?

        Best regards, Karl

        «The Crux of the Biscuit is the Apostrophe»

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1021324]
Approved by davies
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (6)
As of 2014-07-22 10:21 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (109 votes), past polls