Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?

Substituting tokens into configuration values

by dbush (Deacon)
on Mar 20, 2003 at 13:18 UTC ( #244581=perlquestion: print w/replies, xml ) Need Help??

dbush has asked for the wisdom of the Perl Monks concerning the following question:


I appear to be suffering from brain fade today and so I'm having a bit of difficulty starting what I feel should be a simple problem. I wonder if anyone with a working brain can help?

Background: I am trying to write a Win32 service using PerlSvc from ActiveState to achieve something like this SOAP in an NT Service but as I can't predict what I will be running with it later, I wish to have the service driven by a configuration file.

Problem: I wish to enhance my configuration file with "tokens" that refer to other sections of the ini file.

This service has only the following purposes in life:

  1. Read an ini file (Config::IniFile).
  2. Start up a number of processes defined in it (Win32::Process)
  3. Check that the process don't go away over time (Win32::Process)
  4. When the time comes (i.e. the service is cancelled) shut the processes down again (Win32::Process).

The ini looks like this:

[GENERAL] scheduledb=e:\users\dbush\schedule.db [SERVICE] debug=1 interval=5 start1=browser.exe;browser;NORMAL_PRIORITY_CLASS start2=schedule.exe;schedule "localhost:%general-scheduledb%";NORMAL_P +RIORITY_CLASS

The format of the start lines is appname;cmdline;priority.

I have the basics of this, reading the .ini, starting the processes, monitoring then and then shutting them, working well and am happy about this. The problem comes that I wish to extend the command line section to include "tokens" e.g. %general-schedule% that can be parsed to include other configuration items e.g. e:\users\dbush\schedule.db.

After briefly contemplating using one of the templating modules (node Best templating system? and various others) I'm wondering if this is a bit of a sledgehammer approach? Would a better approach be to extract the tokens, retrieve the values and then substitute them back in? I had a Super Search but with little success.

Any help appreciated.


Replies are listed 'Best First'.
Re: Substituting tokens into configuration values
by dakkar (Hermit) on Mar 20, 2003 at 13:36 UTC

    I'd do it like this:

    The INI

    [GENERAL] scheduledb=something [SERVICE] start2=stuff ${GENERAL/scheduledb=default} etcetera

    The "getter":

    sub x_val { my ($ini,$sect,$key,$def)=@_; # same signature as 'val' my $val=$ini->val($sect,$key,$def); my ($sect1,$key1,$def1,$t); while ($val=~/\$\{([^}]+)\}/) { $t=$1; ($sect1,$key1,$def1)=($t=~m|^([^/]+)/([^=]+)(?:=(.*))?$|); $val=$`.(x_val($ini,$sect1,$key1,$def1)).$'; } $val; }


    • It uses pre- and post-match variables, so it will slow you regexps
    • The syntax is a bit different from the one you show
    • It expands recursively
    • It supports in-line defaults
    • I haven't tested it...

            dakkar - Mobilis in mobile
Re: Substituting tokens into configuration values
by zby (Vicar) on Mar 20, 2003 at 13:34 UTC
    For me the simplest approach to configuration files is to use perl syntax - and than directly build the datastructures by an eval. The substitution problem would be solved by an asignemnt in the built datastructure - something like this:
    $config->{general-schedule} = $general-schedule-config
    Where the $general-schedule-config was built by an eval as well.

      You can also use do to accomplish the same thing. Here is a simple example:

      #!/usr/bin/perl use strict; use warnings; use Data::Dumper; our $config; do '/tmp/'; print Dumper(\$config),"\n";

      and /tmp/ looks like:

      $config = [ option1 => 'one', option2 => 'two', option3 => 'three' ];

      It might also be advantagous to setup a signal handler (or whatever means/mechanism available as a win service) that will reread the config.

      "Never be afraid to try something new. Remember, amateurs built the ark. Professionals built the Titanic."
Re: Substituting tokens into configuration values
by Jenda (Abbot) on Mar 20, 2003 at 14:06 UTC

    If you would not insist on Config::IniFile ...

    use Config::IniHash; $config = ReadINI( 'Service.ini', systemvars => 0, # do not expand system %variables% case => 'preserve', # lookup is case insensitive. keys %$INI return the original case # ! if you change this option to a case sensitive one even the %se +ction-name% variables will be case sensitive ! forValue => sub { my ($name, $value, $sectionname, $INI) = @_; $value =~ s/%(\w+)-(\w+)%/$INI->{$1}->{$2}/g; return $value; }, ); use Data::Dumper; print Dumper($config);

    As you see you get a HoH containing the data from the INI file and you can "preprocess" the values before inserting them to the HoH.

    HTH, Jenda

    P.S.: To continue the shameless plug ... did you consider using PerlApp and Win32::Daemon::Simple?

Re: Substituting tokens into configuration values
by hiseldl (Priest) on Mar 20, 2003 at 14:32 UTC

    If you are not stuck using Config::IniFile and can switch to Config::General, then you get variable substitution built in.

    use Config::General; use Data::Dumper; $conf = new Config::General( -ConfigFile => 'y.cfg', -InterPolateVars => 1, ); my %config = $conf->getall; print Dumper(\%config); __END__ # config file contents # GENERAL # scoping allows substituting in the same block or # a nested block, see N.B. below. scheduledb=e:\users\dbush\schedule.db <SERVICE> debug=1 interval=5 start1=browser.exe;browser;NORMAL_PRIORITY_CLASS start2=schedule.exe;schedule "localhost:$scheduledb";NORMAL_PRIORITY_C +LASS </SERVICE> ########### Output: $VAR1 = \%config ### note how $scheduledb is substituted into 'start2' $VAR1 = { 'SERVICE' => { 'start1' => 'browser.exe;browser;NORMAL_PRIOR +ITY_CLASS', 'start2' => 'schedule.exe;schedule "localhost +:e:\\users\\dbush\\schedule.db";NORMAL_PRIORITY_CLASS', 'interval' => '5', 'debug' => '1' }, 'scheduledb' => 'e:\\users\\dbush\\schedule.db' };

    N.B. One thing to watch out for is scoping your name-value pairs. For all the details about scoping for interpolation, check out Config::General::Interpolated.

    What time is it? It's Camel Time!

Re: Substituting tokens into configuration values
by robartes (Priest) on Mar 20, 2003 at 14:03 UTC
    Here's a naieve implementation based on a hash lookup and a regexp.
    #!/usr/local/bin/perl -w my %tokens= ( 'general-scheduledb' => 'e:\users\dbush\schedule.db', 'animal-type' => 'mammal', 'species' => 'camel', ); my $inistring='start2=schedule.exe;schedule "localhost:%general-schedu +ledb%";NORMAL_PRIORITY_CLASS;%species%'; $inistring =~ s/%([^%]+)%/$tokens{$1}/eg; print $inistring; __END__ start2=schedule.exe;schedule "localhost:e:\users\dbush\schedule.db";NO +RMAL_PRIORITY_CLASS;camel
    I fudged your example string a bit to more clearly illustrate the approach.

    Note that this is a naieve implementation - you'll have to add checking that all tokens exist (as this code does not catch typo's in the ini file) for example.

    Update: Renamed %values hash to %tokens. Naming a hash values is asking for a headache :).

    Also, I missed the part that your token values are in the ini file as well. That means that using my implementation, you would build the %tokens hash from the [general] (or whichever) part of your ini file. However, you probably should start looking at modules for this, as jenda suggests below.


Re: Substituting tokens into configuration values
by dbush (Deacon) on Mar 20, 2003 at 17:42 UTC

    Many thanks (and a ++ tomorrow) to zby, crouchingpenguin, dakkar, robartes, Jenda and hiseldl for their insightful replies.

    The solution I have come up with is to use the tie functionality within Config::IniFiles and a slightly modified version of robartes regular expression approach. The while loop handles nested tokens (unlikely as they are in my case).

    The following is a code snippet of the approach I intend to use. Comments welcome.

    #perl -w use strict; use warnings; use Config::IniFiles; my ($appname, $cmdline, $priority, $szProcess); my %config; tie %config, 'Config::IniFiles', (-file => 'service.ini', -nocase => 1 +); my @processesToStart = map { $config{'service'}{$_} } grep { /^start[0-9]+/ } keys %{$config{'service'}}; foreach $szProcess (@processesToStart) { #Break process down into it's parts ($appname, $cmdline, $priority) = split /;/, $szProcess; #Process cmdline while ($cmdline =~ s/%([^%]+)-([^%]+)%/$config{lc $1}{lc $2}/eg) {} #Log and run it already print $cmdline, "\n"; } __END__ Text of service.ini: [GENERAL] scheduledb=e:\users\dbush\schedule.db [SERVICE] debug=1 interval=5 start1=browser.exe;browser;NORMAL_PRIORITY_CLASS start2=schedule.exe;schedule "localhost:%general-scheduledb%";NORMAL_P +RIORITY_CLASS Output: browser schedule "localhost:e:\users\dbush\schedule.gdb"

    The other modules mentioned look interesting (esp. Win32::Daemon::Simple) but I'm afraid I'm up against a bit of a deadline. I also didn't mention that the configuration file must be a standard ini as I share it with some C programs.

    Thanks again,

    Update: Corrected output typo.

      I would suggest one little change to the regexp:

      while ($cmdline =~ s/%([^%-]+)-([^%]+)%/$config{lc $1}{lc $2}/eg)
      This way it should be quicker since it does not force Perl to backtrack after it slurps up the whole general-scheduledb.

      Of course it's not entirely equivalent. If your INI contained %foo-bar-baz%, then your regexp would match it like %(foo-bar)-(baz)% and mine as %(foo)-(bar-baz)%. I don't think it matters.


Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://244581]
Approved by Tanalis
Front-paged by Jenda
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (4)
As of 2022-06-27 11:49 GMT
Find Nodes?
    Voting Booth?
    My most frequent journeys are powered by:

    Results (88 votes). Check out past polls.