Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?

Config Files in Perl

by narashima (Beadle)
on Nov 22, 2006 at 22:03 UTC ( #585624=perlquestion: print w/replies, xml ) Need Help??

narashima has asked for the wisdom of the Perl Monks concerning the following question:

Revered Monks,

I have looking to create an script that needs to gets lots of parameters from a config file.
While researching on the net, I found some very interesting articles.
Below are some approaches I found(or atleast the ones that I understood somewhat):

1.Using a name=value based config file
2.Using a config file with perl code in it and then doing a "require $filename" in the application script to bring in the data
3.Using a config file with perl code in it and then doing a "do $filename" in the application script

I am curious to know if any of you have used these approaches? Any pros and cons for using the above approaches that make using one of the above more favourable?any approaches other than the ones above that are more powerful/better?


Replies are listed 'Best First'.
Re: Config Files in Perl
by GrandFather (Saint) on Nov 22, 2006 at 22:17 UTC
Re: Config Files in Perl
by brian_d_foy (Abbot) on Nov 23, 2006 at 00:22 UTC

    I wrote a configuration file chapter for Mastering Perl. In short, configuration formats with tricky syntax (such as Perl code, XML, or even YAML) cause a lot of problems for human editors, and there are plenty of fine formats which are much more forgiving and plenty of Perl modules to handle them. :)

    brian d foy <>
    Subscribe to The Perl Review
Re: Config Files in Perl
by blue_cowdawg (Monsignor) on Nov 23, 2006 at 01:04 UTC
        I have looking to create an script that needs to gets lots of parameters from a config file.
        any approaches other than the ones above that are more powerful/better?

    Dear Fellow Monk,
    Powerful, appropriate and favorable are all in the eye of the beholder IMHO. I've used many approaches to storing configuraiton information for an application with varying degress of ease, beauty and "power."

    One factor I take into consideration when planning which approach I'm going to use is who is going to be editing the configuration data and how often do they have to do it?

    Quite a few of my scripts are "self configuring" in that they make best guesses about how they are to be configured based on some knowlege of the platform and/or system they are running on. Other scripts run a configuration "wizard" the first time they run and write the configuration somewhere for persistance.

    Other scripts I make the poor user edit a config file by hand. The CPAN module that I like to use for this sort of thing is AppConfig which allows for a very simple syntax for configurtion files making it easy for folks to edit. This is important to me in cases where the user community might not be that sophisticated.

    Another approach I use quite often is to use XML files and use XML::Simple to deal with them. The issue I have here (and has been mentioned elsewhere) is the fact that XML is great for folks that understand how to use it properly but can cause your program to misbehave if the person editing the XML file makes mistakes and creates ill formed XML. A good example of that that I ran into recently is an application that I wrote where I took XML as the input to load entries in a database. The perosn who created the XML file to be used as input put characters into the input that caused problems for the parser such as ampersands and single quotes. Bad ju-ju.

    I've also been known to persist configuration information in a databsse table if my script or application is database driven. This usually means that I either use a wizard for the initial configuration or in one case I wrote an application that had a command line switch that meant "use the values on the command line as defaults and persist them" which is a valid approach no matter what method you use to persist the values.

    There's lots of ways of doing what you're after, but you have to more or less decide in each case what best suits the situation and that you're comfortable using.

    Peter L. Berghold -- Unix Professional
    Peter -at- Berghold -dot- Net; AOL IM redcowdawg Yahoo IM: blue_cowdawg
      Thanks all for your insigtful commnets.
      I believe that using a config file should really be dependent on the user.
      My users are programmers with good knowledge of Perl and so I am going to use a perl based config file and then do a 'do' to get all the config into the script.

      Thanks again

      Perl Rulez!
Re: Config Files in Perl
by xdg (Monsignor) on Nov 22, 2006 at 23:22 UTC

    Unless you expect to users to manage multi-level data structures or complex quoted structures, I'd go with the simplest possible config file, i.e. #1. Personally, I find that Config::Tiny is about as lightweight as can be and I recommend it highly.

    IMO, YAML is useful only if you expect config files to be managed by programmers.


    Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

Re: Config Files in Perl
by traveler (Parson) on Nov 22, 2006 at 22:18 UTC
    Some thoughts:
    • #1 only allows single options e.g name=value. You could use Config::INI::Simple or something similar
    • numbers 2 and 3 can be very dangerous if you make mistakes in them or if people who edit them try to do "bad things"
    • I prefer using XML-based config files and using, say, XML::Simple to parse the file. It is just about as easy as parsing name=value files.
    • You could use Config::General or Config::Generic for more complex configurations, but why not just go with XML?
      I prefer using XML-based config files and using, say, XML::Simple to parse the file. It is just about as easy as parsing name=value files.

      I sort-of second that, except that I'd use YAML. Even if one doesn't know it (I don't really), it should be easy enough to look at some samples and imitate them, or instruct one about how to do so in a few words. It's more human readable and writable IMHO.

        I agree, YAML is great. Loading and saving a config is dead easy, and the config is very human read/writable.
      ... why not just go with XML?

      Because humans might have to modify the configuration file?

        They might. OTOH it might all be done by software. That wasn't specified.

        Now, I agree that XML is not beautiful, but so many people who might edit config files probably are at least somwehat used to HTML, and even vim knows XML highlighting. I do not think it is a problem unless the config file designer goes wild...

Re: Config Files in Perl
by mattr (Curate) on Nov 23, 2006 at 14:46 UTC

    I just wanted to jump in, though the answers are already in the thread I think.

    You could just define a hash at the top of the program, or a __DATA__ section at the bottom, or put settings into a separate module (like I tried those but don't recommend them for setting data values. Exported functions tend to crawl into, and only you really know how to edit it. It requires a lot of discipline to use a separate module for your configuration, and it just confuses everyone else.

    To make software bilingual I've often used a hand-rolled hash which just slurps data in from a two-column text file (skipping comments, and delimited by one or more spaces or tabs). This worked very well, in particular for storing UI error messages and bits of Japanese strings (I'd keep Japanese SJIS-encoded strings in the text file and call them up using an English hash key).

    But for what you usually think of as settings I've had best luck with INI style files, sorry don't remember if it was AppConfig or Config::IniFiles. Lots of people can figure out how to edit them or already know how, and IIRC you can write back to the file too.

    As it happens IIRC (I could be mistaken) the Config::IniFiles module also lets you specify defaults at the top that can be overwritten below. Anyway, I have had good experiences with it, and have used it to roll my own controller for the htdig search engine, to store information on 60 databases and what sites to spider, what languages to let users search in, etc. It is also very nice for small programs, and I've used it with a WxPerl GUI app which also worked quite well. It's been a while but IIRC I kept one copy of the INI file for defaults, and then let the user set defaults in the GUI which got written back to the INI file when they clicked the Save button. This was great.

    Now I am using Catalyst and YAML. Even though YAML sounds easy it took me a while to understand mostly what is going on, and even so I have had to use the command line simulator sometimes. But I think people could figure that out too. It seems that YAML is great for catalyst because by nesting labels you can easily overwrite defaults in several different modules, due to the way it is unrolled on top of the app's shared data structure. So while I have not had long experience with it, it seems that YAML is better for storing small data structures too in your config file, so you get more out of it after you get used to it.

    Much as XML is hyped, I've hated editing XML every time I had to. If humans are going to touch it I would stay away from XML, and go to YAML instead. As it happens, you are also human and it is refreshing to be able to change a config file instead of having to dive into a module and worrying about breaking things, etc.

    Anyway, you may like to experiment a bit but I'd recommend INI or better yet, YAML.

    Oh and here's that code I used for error messages, FWIW. (click the "download" link to view full-width lines). It was useful for making email response templates (munged with HTML::Template) and pull-down menu data like a list of prefectures. I also used it to simulate an apache-style config file for server settings, since you can put in comments. The note that SJIS can break is only true if you use a comma separated list in Japanese and don't use Japanese regex module. Free for you to use, it is quick but maybe better to first learn a little YAML before you miss out...!

    Good luck, Matt

    sub loadtextconfig { # Read a text-based config file into a single-level hash, die +on disk error # Usage: loadtextconfig(\%myhash, $myfilename); # Note: SJIS CAN BREAK/bakemoji e.g. a comma separated list of + prefectures # Copyright 2001-2004 (c) Matt Rosin my ($href,$f) = @_; my ($i,$j) = (0,0); open(W,$f) || die "Content-type: text/html\n\n ERROR: Could n +ot open $f $!"; while (<W>) { next if /^#/; # ignore comme +nted lines $_ =~ s/\r|\n//g; # remove endin +g carriage return and/or ne wline s/^\s+//; # remove leadi +ng whitespace s/\s+$//g; # remove trail +ing whitespace next unless length; # skip blank l +ines ($i,$j) = split(/\t/,$_,2); # $j holds res +t of line $j = "" unless (defined $j); $j =~ s/^\s+//; # remove leadi +ng whitespace $href->{$i} = $j; } close(W); # print "loadtextconfig $f: <PRE>\n" . Dumper($href) . "</PRE> +\n"; }
Re: Config Files in Perl
by pajout (Curate) on Nov 23, 2006 at 13:54 UTC
    My experience is very simple: XML. There is explanation:

    • XML is human readable and machine readable
    • XML is platform independent
    • XML has strict definition of well-formedness
    • There are some tools how to check validity (I prefer RelaxNG)
    • Configuration is, generally, the tree structure of data. XML naturally fits it.
    • There is a lot of mature techniques how to deal with XML - parse it into perl structure directly, DOM, XPATH, XSLT, XInclude...
    Aditionally, when the user is not able to edit XML, he/she is not able to edit other format too - I know, this fact is not so black and white...

    I know that there is some disadvantages, too. But I wanted to describe the advantages.

Re: Config Files in Perl
by doom (Deacon) on Nov 24, 2006 at 20:23 UTC
    In addition to some of the other recommendations here, I thought I'd mention Config::ApacheFormat.

    This let's you create your own configuration files that work much like Apache's: you can have global defaults that can be over-ridden in particular locations. If you're working on something web-oriented, this can be a good choice -- why not use something that's going to be familiar to people working on the project?

    And in general, I think that's the right way to decide the issue: "consider your audience". For example, I would not introduce XML into a project that wasn't already using it for some reason. On the other hand, YAML seems to map pretty well onto the brains of perl programmers, so if the users of your config files are going to be other programmers, than YAML is a good choice...

    But if you're targeting Joe Luser, then you might want something even simpler... though ultimately you're probably going to need to provide some graphical front-end to munge the config file, so machine-readability is even more important -- and that rules out using raw perl code in your config: it's easy for perl to execute, but too hard to parse with another program.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://585624]
Approved by GrandFather
Front-paged by tye
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (6)
As of 2021-01-25 11:46 GMT
Find Nodes?
    Voting Booth?