Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

NestedLoops (Algorithm::Loops) and Iterators

by mrborisguy (Hermit)
on Jul 26, 2005 at 20:59 UTC ( [id://478366]=perlquestion: print w/replies, xml ) Need Help??

mrborisguy has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to generate permutations of a couple lists. After searching here and on CPAN, I've decided that Algorithm::Loops is exactly what I need! Well, almost exactly.

The problem is that instead of having arrays, I have iterators, so I'm wondering if there is any way to use iterators instead of arrays with Algorithm::Loops.

My iterators are used like:

while( defined( my $a = $iter->() ) ) { # do something with $a } #note: $iter->() returns a useful value, or undef

So, I guess what I want is to nest an arbitrary number of while loops instead of an arbitrary number of for loops, conceptually.

I can't just stick everything from $iter into an array, because $iter may be infinite. Does anybody have any ideas on how I could do this?

    -Bryan

Replies are listed 'Best First'.
Re: NestedLoops (Algorithm::Loops) and Iterators
by tmoertel (Chaplain) on Jul 26, 2005 at 22:49 UTC

    One way of looping over nested iterators is to combine them into a single iterator that loops over their Cartesian product. In A mini-language for sequences (part 1), I give the code for seq_prod, which does this for sequences, which are similar to iterators.

    Assuming that your iterators wrap around upon exhaustion, it is easy to convert them into sequences. Then you can combine them with seq_prod and iterate over the resulting sequence. If you want to work with iterators, you can convert the resulting sequence back into an iterator, or you can write an iter_prod function and avoid sequences altogether. In the example code below, I will take the first route.

    Let us say that iterators are defined as follows:

    sub iter { my $vals = \@_; my $i = 0; sub { $i >= @$vals ? do { $i = 0; undef } : $vals->[$i++]; }; } sub enumerate_iter { local $Data::Dumper::Indent = 0; local $Data::Dumper::Terse = 1; my ($iter) = @_; my $i = 0; while (defined($_ = $iter->())) { printf "%2d: (%s)\n", $i++, Dumper($_); } print "\n"; } enumerate_iter( iter( qw[a b c] ) ); # 0: ('a') # 1: ('b') # 2: ('c')

    Then, the following functions let us convert between iterators and sequences.

    sub iter_to_seq { my ($iter) = @_; sub { my $val = $iter->(); defined $val ? ($val) : (); }; } sub seq_to_iter { my ($seq) = @_; sub { my @val = $seq->(); @val ? [@val] : undef; }; } enumerate_iter( seq_to_iter( iter_to_seq( iter( qw[a b c] ) ) ) ); # 0: (['a']) # 1: (['b']) # 2: (['c'])

    Note that we wrap sequences with array references in order to convert them into iterators. (Thus the iterator-sequence-iterator round trip is not equivalent to an identity function.)

    Finally, with the following functions, we can combine iterators as sequences and convert the resulting sequence back to an iterator. The iter_prod function encapsulates the entire process.

    use List::Util qw( reduce ); sub seq_prod { no warnings 'once'; reduce { seq_prod2($a,$b) } @_ ; } sub seq_prod2 { my ($s, $t) = @_; my @sval; sub { my @tval; while ( !@sval || !(@tval = $t->()) ) { return () unless @sval = $s->(); } ( @sval, @tval ); }; } sub iter_prod { seq_to_iter( seq_prod( map iter_to_seq($_), @_ ) ); }

    The following example shows how iter_prod works.

    my $abcees = iter(qw(a b c)); my $xyzees = iter(qw(x y z)); enumerate_iter( iter_prod( $abcees, $xyzees ) ); # 0: (['a','x']) # 1: (['a','y']) # 2: (['a','z']) # 3: (['b','x']) # 4: (['b','y']) # 5: (['b','z']) # 6: (['c','x']) # 7: (['c','y']) # 8: (['c','z'])

    Now you can replace your nested loops with a single loop over the iterator product.

    my $product = iter_prod( $abcees, $xyzees ); while (defined (my $vals = $product->())) { my @vars = @$vals; print "Got vars: @vars\n"; # do real work here } # Got vars: a x # Got vars: a y # Got vars: a z # Got vars: b x # Got vars: b y # Got vars: b z # Got vars: c x # Got vars: c y # Got vars: c z

    I hope this helps.

    Cheers,
    Tom

      That's the same thing as:

      $iter_prod = NestedLoops([ sub { iter(qw(a b c) }, sub { iter(qw(x y z) }, ]);

      Wasn't the whole point to avoid flattening the iterator and to provide arbitrary nesting? Your solution flattens, and the depth is hardcoded as a sequence of calls to iter_prod.

      Note: NestedLoops's iterator returns an array rather than an array ref.

      Update: Your solution does support arbitrary depth:

      my $iter = do { my $done = 0; sub { $done++ ? undef : 1 } }; iter_prod($iter, ...) if ...; iter_prod($iter, ...) if ...;

      With NestedLoops, you'd do:

      my @iters; push(@iters, ...) if ...; push(@iters, ...) if ...;
        Can you define what you mean by "flattening" and "arbitrary depth"? My understanding of these things does not lead me to see unresolved problems in the original post. (Also, I do not believe that the NestedLoops code you provided is equivalent to seq_prod.)

        In the OP's example while loop, the iterator returns scalar values. Thus I don't see how the seq_prod of such iterators could "flatten" the results: there are no intermediate array results to flatten.

        On the arbitrary depth issue, if you mean that iter_prod works only for a fixed number of iterators, that is not the case. You can pass it any number of iterators, and (as long as they are independent) it will yield an iterator over their Cartesian product. If you mean something else, say that the Cartesian product is not what is desired, we can derive a suitable combinator to replace seq_prod2 and use it (with reduce) to build the desired output iterator. For example, we could define a combinator that takes the product of two iterators when the second is generated dynamically from the output of the first.

        Cheers,
        Tom

Re: NestedLoops (Algorithm::Loops) and Iterators
by ikegami (Patriarch) on Jul 26, 2005 at 22:34 UTC

    NestedLoops can't do it. (Correction: NestedLoops can if you pass it tied arrays.) But one could write NestedIters:

    Algorithm/Iters.pm:

    example.pl:

    output:

    NestedIters is identical to NestedLoops. It does everything NestedLoops does, down to the return values and options. However, it has two extra features:

    • Instead of a sub that return an array ref, you may choose to pass a sub that returns an iterator.
    • Instead of a sub that return an array ref, you may choose to pass an array ref.

    This code is a derivative work of Algorithm::Loops.

    Update: The OnlyWhen option is now supported.
    Update: The package is now more aptly named.
    Update: NestedIters now supports subs that return array refs (and just plain array refs), making it drop-in replacement for NestedLoops. It will automatically create iterators from array refs.

Re: NestedLoops (Algorithm::Loops) and Iterators
by runrig (Abbot) on Jul 26, 2005 at 22:19 UTC
Re: NestedLoops (Algorithm::Loops) and Iterators
by Solo (Deacon) on Jul 26, 2005 at 22:58 UTC
    While NestedLoops doesn't handle iterators for the values to loop over, it seemed easy enough to me to implement one that does--minus error checking, code wrapping and other conveniences that NestedLoops provides.

    Update: well, I only lost by 24 minutes. lol

Re: NestedLoops (Algorithm::Loops) and Iterators
by AReed (Pilgrim) on Jul 26, 2005 at 22:18 UTC
    I'm confused although others here may not be. It would help me if you could give a simple example of input and output and an explanation of how your iterator works. The code would be great, if short enough. Given my understanding of the word "permutation", I can't get past the statement that an iterator may be infinite...

      Okay, say I have three iterators, $iter1, $iter2, $iter3. I'll demonstrate an example of each by just using them:

      while( defined( my $item = $iter1->() ) { print "1: $item\n"; } while( defined( my $item = $iter2->() ) { print "2: $item\n"; } while( defined( my $item = $iter2->() ) { print "3: $item\n"; } #outputs: 1: 1 1: 2 1: 3 2: y 2: n 3: 7 3: 14 3: 21 3: 28

      So, what I want to do is something like:

      $nl_iter = NestedLoops( \@Loops ); # some fancy magic for \@Loops while( my @items = $nl_iter->() ) { print join ", " @items; print "\n"; } #outputs: 1, y, 7 1, y, 14 1, y, 21 1, y, 28 1, n, 7 1, n, 14 ... 3, n, 21 3, n, 28

          -Bryan

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://478366]
Approved by kwaping
Front-paged by friedo
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others goofing around in the Monastery: (5)
As of 2024-04-24 12:48 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found