Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change

Plug-in architectures

by Masem (Monsignor)
on Mar 14, 2002 at 01:33 UTC ( #151572=perlmeditation: print w/replies, xml ) Need Help??

People have brought up the concept of plug-ins in perl, such as at Plug-in mechanism implemented in Perl (with the first response giving other searches). The concept here is that code is self-contained in a separate .pm file but located outside of @INC. However, in the FBP system I've been looking at, I'm considering a plug-in system that operates in a slightly different fashion, and based on what I've done with Java in the past.

First, plugins make a bit more sense here than trying to have everything drop into @INC. Since not every plugin could be used by the user at any time, it's probably smarter to keep the call table clean and only load modules when needed. In addition, it would seem to be much more easier for the user to manage without adminstrator help (though normal perl mods should also work the same way).

However, I envision that some of the plugins that could be written would require additional files, either in the way of extra perl code, glue libraries, config files, text files, or whatnot. Again, these could all be installed into @INC without problem (and be referred to), but now you start to get a messy plug-in directory.

If you look at (at least early versions of) Netscape, Photoshop, etc, plugins were single files; on the Mac side, this was 'easily' accomplished using the resource fork, and I believe similar tricks were used for the PC side. When I tried this in Java, I used .jar files to group class files and other files into a single package. The natural extention in perl is tar.gz files. Inside the tar.gz. file would be some manifest file (say, XML based) that would tell the program where the key class is and any other important files might be located.

The idea would be that at startup of the program, the various directories that have been indicated as plugin dirs (including a program default, system local, and user local set) would be scanned; each file in those dirs would be tested as a tar.gz file, then opened to look for a manifest file (as by a fixed name). At this point, the plugin name and location would be set up into a global hash. At some point, when the plugin is needed, then the necessary code is dynamically loaded using the location info and temporary strings as obtained from the tar.gz file. Plugin authors would have the ability to grab any other data files from their plugin archive via provided code.

Does this approach seem reasonable, has it been done before, or does it seem over the edge for what it needs to do and in this case just rely on @INC to provide what's need? The only problem of this latter solution is that how one determines which files are plugins for the system in question or not.

Dr. Michael K. Neylon - || "You've left the lens cap of your mind on again, Pinky" - The Brain
"I can see my house from here!"
It's not what you know, but knowing how to find it if you don't know that's important

Replies are listed 'Best First'.
Re: Plug-in architectures
by abstracts (Hermit) on Mar 14, 2002 at 02:41 UTC

    The idea seems interesting as it allows installing plugins (or modules) by simply downloading them to a specific directory (/usr/share/perl/something or ~/.perl_modules). However, I don't like the idea of scanning all tgz files in these directories at runtime as it seems to would kill startup time.

    I think the application should scan plugin directories for plugin names to build a hash of available plugins. Later, when a plugin is actually needed, the xml or whatever file would be located, dependencies are checked and files are loaded. Runtime errors (or warnings) can occur if necessary.

    Following this track, I think the file names could also contain an optional version number because an application may require a specific version of a plugin.

    Could plugins or modules conflict with one another? If they can, then you may be on your way to a more sophesticated mechanism for implementing packaging (it might look like deb or rpm).

    Dunno what others think about this and I hope I didn't miss an important point.


Re: Plug-in architectures
by perrin (Chancellor) on Mar 14, 2002 at 04:13 UTC
    IMHO, total overkill. Look for plugins in @INC. It's easy enough for people to change PERL5LIB if they need to. Make them all have a common naming structure like MyProject::Plugin::PluginName. That's how Template Toolkit does it, and it seems to work very well.
Re: Plug-in architectures
by zengargoyle (Deacon) on Mar 14, 2002 at 01:59 UTC

    Sounds cool. I'm not sure of tar's file format. I'm assuming that it's something like

    [ [ <tar file header> ], [ 'path','file','perm','data'], [ <another file> ], [ <yet another> ], [ <tar file done> ] ]

    Because tar ztvf file.tar.gz takes too long for there to be a full directory at the beginning of the tarfile. So I'm thinking, enforce the manifest to be the first file in the archive. I wouldn't want to have to uncompress a huge archive because the manifest was the last file and not the first.

    I'm pretty sure tar can do this, don't know about plugin creators ;)

    Or I could be wrong about tar... Just a thought.

Re: Plug-in architectures
by tomhukins (Curate) on Mar 14, 2002 at 08:58 UTC

      bzip2 is another alternative for compressing files.

      Update: I mentioned bzip2 for completeness, not as advocacy. tomhukins' points below are quite valid.


        bzip2 is great if your aim is to compress files as small as possible. It's much slower than gzip and ZIP encoding, though, so for files which are frequently read from or written to, as we're dealing with here, it's inappropriate. Also, bzip2 is just a compression mechanism: you'd still need to use something like tar to archive multiple files into one file.

        See the Open Office document I mentioned earlier for an explanation of why .tar.gz encoding isn't the best approach, then add the comparative slowness of bzip2 (especially for compressing files), and you've got a solution which is far from optimal.

Re (tilly) 1: Plug-in architectures
by tilly (Archbishop) on Mar 15, 2002 at 06:17 UTC
    If you do this, then I would suggest making the plug-in architecture itself pluggable. :-)

    Note that many people have implemented plug-in type structures where they store the code in a database table. The same can be done with dbms as well. Abigail has the amazing module described here which can be used to pull down plugins from the network.

    I see nothing wrong with being able to mix and match at will...

    (Not that I would see a need for this. My needs are met by existing solutions. I just think this would be kind of neat to see.)

Re: Plug-in architectures
by AidanLee (Chaplain) on Mar 14, 2002 at 17:47 UTC

    Sounds like a pretty good idea to me, however i have to wonder: why both tar and gzip? It seems to me that your goal is just to get all the information (files, manifest, etc) packed into a single file, and tar works fine on its own. gzip will give you smaller files at the tradeoff of the time it takes to uncompress (and recompress?) them. So unless size is a major concern (probably not when dealing with perl source code), i see it as unnecessary.

    You could also consider that there's no requirement for your files to end in '.tar' if you're using them solely for the purpose of acting as plugins for your program. if you give them a more distinctive extension (.ppi for perl plug-in?) it will take some of the guesswork out of figuring out which files are plugins and which are garbage in any given directory.

      You're probably right, tar along would most likely be better than tgz. (It's very common to always see tar and the associated gzip together, so sometimes it takes a bit of insight to remember that they can be separated :-)

      And yes, I realize that there's no need to stick to .tar; perl could care less what that extention is.

      I think , as tilly suggested below, is that it might be best to start with a general plug-in system that allows people to handle what they want. (Plugins to that? hmmm, let's see what I can come up with... :-) ). Of course, I can see an initial step, that being of writing a hash tied to a tar archive (keys being filenames). While with this, there's issues with setting keys, assuming that one is simply reading from a tar, that would be a good way to store and retrieve the tar-stored files when needed.

      Dr. Michael K. Neylon - || "You've left the lens cap of your mind on again, Pinky" - The Brain
      "I can see my house from here!"
      It's not what you know, but knowing how to find it if you don't know that's important

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://151572]
Approved by root
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (5)
As of 2018-06-22 16:59 GMT
Find Nodes?
    Voting Booth?
    Should cpanminus be part of the standard Perl release?

    Results (124 votes). Check out past polls.