Beefy Boxes and Bandwidth Generously Provided by pair Networks
Come for the quick hacks, stay for the epiphanies.

Re: Config file to tree format with id, parent, key, value

by choroba (Cardinal)
on Nov 01, 2016 at 22:51 UTC ( #1175088=note: print w/replies, xml ) Need Help??

in reply to Config file to tree format with id, parent, key, value

If you can't find a module to parse your config format, write your own parser. Marpa::R2 can help you in the task:
#!/usr/bin/perl use warnings; use strict; use feature qw{ say }; use Marpa::R2; my $input = << '__INPUT__'; # This is a comment feed_realtime_processor_pool = ( 11, 12 ) ; dropout_detection_time_start = "17:00"; # Sometimes the config can have sub-structures named_clients = ( { name = "thread1"; user_threads = ( { name = "realtime1"; cpu = 11; } # more comments { name = "realtime2"; cpu = 12; } # more comments ); } ); __INPUT__ my $dsl = << '__DSL__'; lexeme default = latm => 1 :default ::= action => ::first Config ::= Elements Elements ::= Element+ action => grep_def +ined Element ::= (Comment) action => empty | Name (s eq s) Value action => [values] | Name (s eq s) Value (semicolon s) action => [values] Comment ::= (hash nonnl nl) action => empty Name ::= alpha Value ::= List | String | Num | Struct List ::= (lpar) Nums (rpar s semicolon s) Nums ::= Num+ separator => comma action => listify Num ::= (s) digits (s) | (s) digits | digits (s) | digits String ::= (qq) nqq (qq semicolon s) action => quote Struct ::= (lpar s) InStructs (rpar semicolon s) InStructs ::= InStruct+ action => grep_def +ined InStruct ::= (lcurl s) Elements (rcurl s) | (Comment s) action => empty | Element s ~ [\s]* eq ~ '=' hash ~ '#' nonnl ~ [^\n]* nl ~ [\n] alpha ~ [a-z_]+ lpar ~ '(' rpar ~ ')' lcurl ~ '{' rcurl ~ '}' semicolon ~ ';' comma ~ ',' digits ~ [\d]+ qq ~ '"' nqq ~ [^"]+ __DSL__ sub listify { shift; [ @_ ] } sub quote { qq("$_[1]") } sub empty {} sub grep_defined { shift; [ grep defined, @_ ] } my $id = 1; sub show { my ($parent, $name, $elems) = @_; if (ref $elems->[0]) { show($parent, $name, $_) for @$elems; } elsif (ref $elems->[1]) { if (ref $elems->[1][0]) { say join ', ', $id, $parent, $elems->[0], 'null'; show($id++, $elems->[0], $elems->[1]); } else { for my $e (@{ $elems->[1] }) { say join ', ', $id++, $parent, $elems->[0], $e; } } } else { say join ', ', $id++, $parent, @$elems; } } my $grammar = 'Marpa::R2::Scanless::G'->new({ source => \$dsl }); show('null', q(), ${ $grammar->parse(\$input, 'main') });


1, null, feed_realtime_processor_pool, 11 2, null, feed_realtime_processor_pool, 12 3, null, dropout_detection_time_start, "17:00" 4, null, named_clients, null 5, 4, name, "thread1" 6, 4, user_threads, null 7, 6, name, "realtime1" 8, 6, cpu, 11 9, 6, name, "realtime2" 10, 6, cpu, 12

Note that the id of the last line is 10, not 9 as in your sample. Adding leading zeroes and aligning the columns left as an exercise for the OP.

($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord }map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,

Replies are listed 'Best First'.
Re^2: Config file to tree format with id, parent, key, value
by arthur_accioly (Initiate) on Nov 02, 2016 at 01:19 UTC
    Wow, it's working!!!! Thank you a lot. Just an addendum, to install this module I had to install a lot of extra dependencies that were not declared:
    sudo cpan IPC::Cmd sudo cpan Module::Build sudo cpan Time::Piece sudo cpan Marpa::R2
    Thanks again!

      IPC::Cmd and Time::Piece have been in core since 5.9.5 which is very nearly a decade ago. Many dists don't mention core modules as dependencies, perhaps because the authors (erroneously) assume that everyone has the core installed by default. Module::Build used to be in core and it's possible that the author of Marpa::R2 did not notice when it was dropped from core.

      Which version of Perl are you running and on which operating system?

        Hi, this is perl 5, version 16, subversion 3 (v5.16.3) built for x86_64-linux-thread-multi. I'm using centos7 but can use 6 as well. It's not a problem to install those extra modules. Thanks.
Re^2: Config file to tree format with id, parent, key, value
by ron.savage (Acolyte) on Nov 04, 2016 at 04:50 UTC
Re^2: Config file to tree format with id, parent, key, value
by arthur_accioly (Initiate) on Nov 07, 2016 at 13:45 UTC
    Hi Choroba, can I ask you one more thing? Those config files can come in many different formats. I have two questions: 1. The order where the entries are coming is important? For example, if "dropout_detection_time_start" comes before "feed_realtime_processor_pool", is this going to break my parser? 2. If we have situations where some of the rules are not present in the config file, is this going to break the parser? For example, if I have "another_realtime_processor_pool", is this going to be discarded? Obviously I'll create a grammar for those extra parameters, but there are situations where the configs can have different entries depending of what the user wants to put in the config. Thanks.
      1. The order shouldn't be important in this case. You can see that both "dropout_detection_time_start" and "feed_realtime_processor_pool" are instances of Element, that combine to Elements in any order ( Element+ ).

      2. Have you noticed that the parser is universal? It doesn't look for "realtime_processor" anywhere, it searches for Name , so you can define anything you like (however, at the same time, it means the parser doesn't catch typos - if you have the exhaustive list of possible Names available, replace the general alpha by something more specific).

      ($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord }map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1175088]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others meditating upon the Monastery: (4)
As of 2023-10-04 13:40 GMT
Find Nodes?
    Voting Booth?

    No recent polls found