iterators: traversing arbitrary data structure

[dimar]

Given a data structure like this ...

$oWkBook    = {
    students =>[
        {fname=>'joe',lname=>'bloe',age=>'18'},
        {fname=>'jill',lname=>'bill',age=>'16'},
    ],
    teachers => [
        {fname=>'joe',lname=>'sloe',age=>'38'},
        {fname=>'jill',lname=>'sill',age=>'36'},
    ],
    admins  =>[
        {fname=>'koe',lname=>'moe',age=>'58'},
        {fname=>'dill',lname=>'mill',age=>'56'},
    ],
};

How can I change this ...

### iterate (current)
for my $oWkSheet (keys %{$oWkBook}){
    for my $oRec (@{$oWkBook->{$oWkSheet}}){
        for my $oFld (keys %{$oRec}){
            $oRec->{$oFld} =~ s/^\s+\n$//;
        };
    };
};

into this ...

### iterate (wanted)
for my $item ($oWkBook->iterate_fields()){
    $item   =~ s/^\s+\n$//;
};

I've given SuperSearch a say, but it seems the examples assume a simple sequence of items (e.g. array), when what I want to do is iterate over an arbitrarily-nested data structure. (e.g. HoAoH)

My long term goal is to be able to reproduce this pattern regardless of the structure of the data variable. The request here is for a link or an example of handling HoAoH with iterators that I can use as a starting point for understanding this design pattern a little better.

[dragonchild]

Look at how Data::Dumper does it. Essentially, you will want to use Scalar::Util's reftype() and go from there.

[ikegami]

Update: OOPS! The solution in this post will show you all the values, but will not allow you to change them. See my reply to this post for a more appropriate solution.

Read more... (2 kB)

[ikegami]

You could do

my $iter = get_value_iter($oWkBook);
while ($iter->(my $item_ref)) {
   $$item_ref =~ s/^\s+\n$//;
}

if you defined get_value_iter as follows:

sub get_value_iter {
   our $val; local *val = \$_[0];

   if (ref($val) eq 'SCALAR') {
      my $ref = $val;
      return sub {
         my $sub_iter = get_value_iter($$ref);
         return $sub_iter->($_[0]);
      };
   }

   if (ref($val) eq 'ARRAY') {
      my $ref = $val;
      my $key = 0;
      my $sub_iter;
      return sub {
         for (;;) {
            return 1 if $sub_iter && $sub_iter->($_[0]);
            return if $key >= @$ref;
            $sub_iter = get_value_iter($ref->[$key++]);
         }
      };
   }

   if (ref($val) eq 'HASH') {
      my $ref = $val;
      my @keys = keys %$ref;
      my $sub_iter;
      return sub {
         for (;;) {
            return 1 if $sub_iter && $sub_iter->($_[0]);
            return if not @keys;
            $sub_iter = get_value_iter($ref->{shift @keys});
         }
      };
   }

   {
      my $ref = \$val;
      my $done = 0;
      return sub {
         return if $done;
         $_[0] = $ref;
         $done = 1;
         return 1;
      };
   }
}

Scalar::Util's reftype can be used instead of ref if you wish to traverse objects.

[eric256]

Hey, Not an iterator, but it works.

use strict;
use warnings;

use Scalar::Util qw(reftype);
use Data::Dumper;


my $oWkBook    = {
    students =>[
        {fname=>'joe',lname=>'bloe',age=>'18'},
        {fname=>'jill',lname=>'bill',age=>'16'},
    ],
    teachers => [
        {fname=>'joe',lname=>'sloe',age=>'38'},
        {fname=>'jill',lname=>'sill',age=>'36'},
    ],
    admins  =>[
        {fname=>'koe',lname=>'moe',age=>'58'},
        {fname=>'dill',lname=>'mill',age=>'56'},
    ],
};

print reftype($oWkBook);
my @test = getScalars($oWkBook);
for (@test) {
    $$_ .= "test";
}
my $link = \$oWkBook->{students}->[0]->{fname};
print $link;
$$link .= "test";

print Dumper($oWkBook);
print Dumper(@test);

sub getScalars {
    my $it = shift;
    my $type = reftype($it);
    my @out;
    if (defined $type && $type eq "HASH") {
       for my $key (keys %$it) {
           if (!defined reftype($it->{$key})) {
              push @out, \$it->{$key};
           } else {
              push @out, getScalars($it->{$key});
           }
       }
    } elsif (defined $type &&  $type eq "ARRAY") {
       for  ( @$it ) {
           if (!defined reftype($_)) {
              push @out, \$_;
           } else {
              push @out, getScalars($_);
           }
       }
    } elsif (defined $type &&  $type eq "SCALAR") {
       push @out, $it;
    } else {
       return \$it;
    }
    return @out;
}

Left some of my fumbling in there, you can feel free to ignore that part. ;)

[BrowserUk]

Take a look at Data::Rmap. It's rmap() routine is a recursive map specifically for traversing nested data structures.

use Data::Rmap;

...

rmap{
    s/^\s+\n//;
} $oWkBook;

[CaMelRyder]

Try using ref() to determine what is stored at each dimension. Then you can make the decision about how to iterate. I believe that this subroutine will do the trick, however, I am creating this off the top of my head, so please don't be afraid to criticize.

sub recIterate{
   my $ref = shift;
   my $funcRef = shift; #mutation function reference

   if(ref($ref) =~ /HASH/){
       my %copy = %$ref;
       foreach my $item (keys %copy){
           $copy{$item} = recIterate($copy{$item}, 
                                     $funcRef);
       }
       %$ref = %copy;
   }elsif(ref($ref) =~ /ARRAY/){
       my @copy;
       foreach my $item (@copy){
           push(@copy, recIterate($item, $funcRef);
       }
       @$ref = @copy;
   }elsif(ref($ref) eq ''){ #i think this is how scalars 
                            #look to ref, but you might
                            #want to look it up
       $$ref = &$funcRef($$ref);
   }
}

In order to use this you must define what happens to each scalar value that is in the data structure with a subroutine that takes one scalar as a parameter and returns the new one. Just pass a reference to it when you call recIterate. Here is a usage example:

#you will need to define your own mutator function
sub someFunc{ #removes tabs
    my $old = shift;
    my $new = $old;
    $new =~ s/\t//g;
    return $new;
}

%nDimHash; #this might be a HoAoHoAoAoAoS or whatever;
%nDimHash = recIterate(\%recIterate, \&someFunc);

That should do the trick. Each dimension that is a hash or an array is rebuilt at that level and the previous dimension is is set to point to the new one. The garbage collector should clean up after us. Each scalar value in the example will have all tabs removed, but you can make the function do whatever you want. One more word of caution, this will take some major overhead if you want to iterate a deep data structure. I hope this does the trick. ¥peace out guys, thanks for reading my first post, CaMelRyder¥