dynamic hash nesting

[karrakis]

This may fall in the dangerous category of "why using variables to name variables is stupid" (yes, I've read the thing).

Then again, it may not. I'm going to be very explicit about the objective of all this so I don't get someone shoving a sword through my belly immediately, but what I really need to know is not how-to-do-all-the-things but just how to do this:

I need to dynamically create a nest of hashes from a text file with data separated by tabs and newlines.

example:

Assault    Assault
    Battery    Assault and Battery
    Sexual    Sexual Assault
    Deadly    Assault with a deadly weapon
        with    Assault with a deadly weapon
        without    Assault without a deadly weapon
    Aggravated    Aggravated Assault

I want to pull such a text file (of depth unknown) into a (potentially indefinite) nest of hashes that can be used to derive outputs from the data without including the outputs explicitly in the code.

The size of the file, and depth of the contextual relationships contained within, will vary. The number of tabs before a word or phrase will dictate its depth in the hash, and a new hash tree will be created by any word or phrase with no tabs preceding. Any $value followed by a newline and more tabs than preceded it should become a hash itself, where any $value followed by a newline and the same number or fewer tabs as preceded it should become a scalar (will be treated as output values)

It is important that people (not necessarily me) be able to modify the desired outputs ("Assault and Battery", "Assault with a deadly weapon" etc - i.e.) in the text file, rather than in the code.

Is this something that is possible in Perl?

Thank you for indulging my nubcakery

[moritz]

Is this something that is possible in Perl?

Of course.

The usual strategy to build a data structure of arbitary depth is the "create as you traverse" approach, ie you just walk level by level, create them if needed, and always keep a reference to the previous level to start the current iteration from.

Here you don't even need to do that, because you create at most one level at a time.

Since you didn't explain exactly how the resulting data structure should look like, I've added the description as an extra key called DESCRIPTION.

Without further ado, the code:

use 5.010;
use strict;
use warnings;

my %h;
my @depth;
while (<DATA>){
    chomp;
    /^(\s*)/;
    # 'cause it's 4 spaces here, not tabs
    my $level = length($1) / 4;
    s/^\s+//;
    my ($key, $val) = split /\s{2,}/, $_, 2;
    my $h = $level == 0 ? \%h : $depth[$level - 1];
    $h->{$key}{DESCRIPTION} = $val;
    $depth[$level] = $h->{$key};
}

use Data::Dumper;
print Dumper \%h;


__DATA__
Assault    Assault
    Battery    Assault and Battery
    Sexual    Sexual Assault
    Deadly    Assault with a deadly weapon
        with    Assault with a deadly weapon
        without    Assault without a deadly weapon
    Aggravated    Aggravated Assault

Here is the output, with reduced indentation for better readability:

VAR1 = {
    'Assault' => {
        'Sexual' => {
            'DESCRIPTION' => 'Sexual Assault'
        },
        'Battery' => {
            'DESCRIPTION' => 'Assault and Battery'
        },
        'DESCRIPTION' => 'Assault',
        'Aggravated' => {
            'DESCRIPTION' => 'Aggravated Assault'
        },
        'Deadly' => {
            'DESCRIPTION' => 'Assault with a deadly weapon',
            'without' => {
                'DESCRIPTION' => 'Assault without a deadly weapon'
            },
            'with' => {
                'DESCRIPTION' => 'Assault with a deadly weapon'
            }
        }
    }
};

To look up elements of arbitrary depth, you can use something like

sub lookup {
    my $h = \%h;
    $h = $h->{$_} for @_;
    return $h->{DESCRIPTION};
}

say lookup 'Assault';
say lookup qw/Assault Deadly with/;

Which again drags a reference from the lookup of the previous level to achieve arbitrary depth.

[karrakis]

Thanks, that broke a block that's been bothering me for three days, and probably saved me a week of optimization work, at least.

[sundialsvc4]

Welcome to PerlMonks...   That, believe it or not, is how things are ’round this here neck of the Internet.   (And that’s how fast you can get a really great answer to what you thought was an “off the wall” question.)   You’ll get used to it.

[GrandFather]

If you have control over the file format you may like to consider using something like YAML or JSON instead of a hand rolled configuration file format.

[sundialsvc4]

One thing that comes to mind right now is that you probably need to vet that data before you build algorithms that (necessarily...) rely upon your assumptions about it.   Specifically, do you know that the beginning of each and every line consists of 0 .. n ASCII "Tab" characters?   No spaces?   Do you know that the number of tabs in any line is no more than 1 greater than the number of tabs on the line preceding it?   Do you know that the first line contains no preceding tabs?

I suggest that the first part of your processing sequence should be a script whose sole purpose is to validate these assumptions about the data format.   Garbage in = Garbage out, and only the computer can tell you if it’s garbage.   If the data is not proved to be clean, then do not process it.   If the data is proved, then you can write algorithms based on any assumption that you have proved ... and yet, those subsequent algorithms should also be suspicious.   (Maybe the data wasn’t vetted this time, and an error has crept into the data from somewhere upstream ... only the computer is in a position to sound the alarm.)

Regular-expressions can easily handle these tests.   Each and every line must consist, anchored at the beginning of line, of zero-or-more tabs followed by a non-blank character.   If you extract the tabs in a group, the length of that string is of course the number of tabs.

In the future, I think that you definitely need to move this data to an alternate format, either XML or JSON.   Both of these are well-understood formats for representing hierarchical data, and in the case of XML, have formally-defined validation engines that do not require the writing of source-code.

[karrakis]

good advice, easily implemented. I shall do this thing.