Module configuration for user vs. developer

[bibliophile]

Brethren and Sistern,

I find myself in a design quandry, and seek enlightenment...

I've built a database-abstraction-layer module for handling a particular industry-specific data type (bibliographic MARC records). This abstraction layer, of course, uses DBI under the hood.

I've built a few different modules that the abstraction layer would use to connect to different back-ends (MySQL, Postgres, flat files,...). These use DBD.

This is all so that an application can just say "save this horribly complicated binary data thingy" without worrying about it. It works really well.

Right now, I'm hard-coding into the abstraction layer which particular back-end interface to use. Eventually I'll have that pulled from a config file of some sort.

I've got a bazillion tests - abstraction layer and back-end modules.

But here's my question (finally!): Is there some standard way figuring out (at "make" time? Before then?) which back-end the user needs/wants? (eg: if they have both Postgres and MySQL installed, but only have rights to use one or the other).

The second part of this question is for me as a developer - I've got all of the various databases/data stores running, and want to have the "make / make test" run against all of them. Again, right now I manually change (in the abstraction layer) which backend gets used, wrap the unused back-end tests in skip blocks, and then re-run the make. Is there a better way? How would you do it?

Any help you could give would be... well... helpful. :-)

Update:Thanks, all, for your thoughts and your reasoning - it is much appreciated!

[Tanktalus]

Take a look at how DBI, CGI::Session, and other plug-in-backend style modules do this. They don't figure out anything at make time, they figure it out at runtime. And they figure it out at runtime because someone passed in a string with the backend to be used.

Your tests could then loop through the backends and test each one. And then your "make test" run will test all of them.

Hope that helps,

[bibliophile]

I'm of two minds about this. On the one hand, it seems reasonable that an app pass in some config information.

On the other hand, in an ideal world the app wouldn't care which backend is being used (which would be one of the strengths of the abstraction layer).

Clearly I need to think about this some more :-)

[Tanktalus]

What passing in configuration information means is that the base application can request the same configuration information in whatever means it wants to from the user (commandline parm, hardcode, config file, whatever). For example, if you had a dot-rc file (e.g., ~/.yourapprc) where this info was stored, the user would tell you once, and never have to worry about it again.

No matter what, the user will likely have to tell you something. Interrogating all known databases on the system (which may include TB-sized databases that are remote) might be seen as uncouth by some ;-) So, whether they tell you during the Makefile.PL process (bad idea - makes it difficult to automate the install unless it's a commandline parm) or via a config file (easy to automate), they tell you something.

Separating out your configuration from the app's configuration is also a good thing. You may write all this code to read a config file, but then find out that the app is using a different config file, of a different format (e.g., YAML or XML). Then you'll just look clunky. By forcing the app to pass in your configuration information, you'll allow the app to define its own configuration format and file, and you just get to piggyback on that.

Note that it's very rare that an abstraction layer completely removes the app's care about which backend is used. What an abstraction layer is supposed to do is limit that care to a single line - which, as in this case, might be passed in via config file. I've worked with many abstraction layers, and I've never seen one smart enough to know what I want without me telling it first ;-)

[Joost]

You should probably use ExtUtils::MakeMaker's prompt() function in your Makefile.PL to ask the user what (s)he wants to install.

For instance:

use ExtUtils::MakeMaker;

my @dbds = qw(DBD::mysql DBD::Pg); # whatever
my @installed = grep { eval "require $_" } @dbds;

my $answer = prompt ("I can use the following drivers: @dbds. 
You have the following drivers installed: @installed
Please type the drivers you want to test, seperated by a space:","@ins
+talled");

my @use = split / /,$anser;

open  F,"> dbds_for_testing.pl" or die $!;
print F "\@DRIVERS = qw($answer);";
close F;

WriteMakefile(
    # ....
    'PREREQ_PM'     => {
        map { $_ => 0 } @use
    }, # e.g., Module::Name => 1.i

    # ....
);

And you can then do "dbds_for_testing.pl" in your test code to see which drivers you need to test and skip the others.

[bibliophile]

Oooh... that looks like fun!

Thanks, Joost. I'll monkey with that for a while :-)

[herveus]

Howdy!

Makefile.PL (or the functional equivalent if you are using Module::Build)

You can interact with the user when they do "perl Makefile.PL". For example, you could ask them which back-end to use and pass that information on somehow. Can you abstract out the choice of back-end into a simple configuration item? Alternately, you might be able to mechanically discover answers (or potential answers to confirm with the user).

Makefile.PL can do a lot of stuff when it's necessary. You just write suitable code, although the implementation details can be "interesting". I did this once a while back, so I'm fuzzy on the details, but I know that you can do it. You might also look at modules that interact with you when you run 'perl Makefile.PL' and steal ideas.

[bibliophile]

Absolutely, the choice of back-end can become a simple config item. I'm a little (ok, more than a little) makefile-challenged, so I need to hit the books on this.

The further bit of complication is that whichever data store is used must have a bit of pre-configuration done before the test suite can run (hmm... maybe I can set up a "test" that does this, which can run as the first test....).
This would make the pick-one-at-make-time a good option.