Toolset configuration

[mrhenroi]

I'd like to know how others handle the configuration of their tool sets. My group maintains a set of tools that are configured with configuration files written in Perl. Each configuration file contains one or more hashes which can then be used in each tool.

# configs_topic.pl
package Configs;

%some_hash = (
  key1 => "value1",
  key2 => {
    key3 => 'value2',
  }
);

# my tool
require "configs_topic.pl";
my $value = $Configs::some_hash{key1};

A few of my many problems:
1. The configuration files are becoming way too huge. It seems like we try to configure everything. How much is too much?
2. If you change the configuration file data structure, you may affect multiple tools and not just the one you're updating.

[Tanalis]

The way we handle this in the shop I work for is to have a global, sitewide configuration that's split up into logical segments based upon what each part of the configuration is for.

When writing new stuff, we have a Config module set up that allows configuration segments to be loaded independently of one another, in the combinations that we usually use them in (for convenience) or all at once (for when you can't find the bit of the config database that you need *grin*).

By abstracting out the configuration layer, we ensured that the new code (and any old code) has no knowledge of what the data looks like on disk (it's actually XML at the minute): it simply gets served up a configuration hash from the config layer. If something about the file's structure needs to change, or even we move from a flat file to some other storage method, the change impacts only the config layer - no other code needs to be modified.

As far as "how much is too much" goes, I'm a big fan of reusable, configurable code, but not at the expense of readability and ease of maintenence. The time to stop adding configuration variables is when the code becomes less clear by using them, or becomes harder to maintain owing to the need to track down a load of configuration data, often split over a number of files, simply to see what a section of code's doing.

[themage]

Hi,

I do this using XML files, that I read with XML::Simple and a Perl Module that handles some XML tags, like <include file="someother.xml"/>

Then I split my configs by several files, with options specific to some thing, and used by all needed parts.

Then I make config files to the tools that just include the splits each tool need.

I try to change the splits as little as possible, and usually just to add new options. When I need to restructure the options tree to one of the tools, I just restructure the main xml file for that particular tool, repositioning the includes (which are just the leafs in each branch).

This is for sysadm/offline tools. When talking about websites, I use a table in the site database where all options are saved, and where I can edit them.

[anonymized user 468275]

Or...

The object-oriented model fits this scenario. Your config package would load the config files into what will later be private (local) scope (during the change process you have to let old code break that rule) and later hand it out to tools via new methods that have parameters as necessary to fully specify the configuration value being requested.

You set about modifying your tools to use the new methods instead of accessing the data structure directly. Only when that process is complete have you made your tools independent of data structure.

[Anonymous Monk]

I don't have a fixed way of using a configuration style. It all depends. I don't not like to use Perl hashes, mainly for two reasons:
• It only works for Perl programs. I also use tools, written by myself or by others, that were written in different languages, with shell, C, Pascal and Java the main languages being used here.
• There's too much syntax surrounding hashes. All needed for a Perl program of course but it gets in the way for a configuration file. Specially if it's a configuration file that might be modified by someone who isn't familiar with Perl.
I use configuration files written in the shell (for setting environment variables), written in Apache format (for parsing with Config::General), written in inifile formate (for parsing with Config::IniFiles), a few YAML files (a mistake I won't repeat - YAML files are human readable, but editing remains tricky), and lots of home grown formats with dedicated parsers. It may be more work to write the parser (but often, the parsers can be quite simple), but shear ease of writing the configuration file pays off.