Great, as long as you don't want to generate all 10-element subsets of a 20-item list. Or 45-choose-20. In those cases, you will need an iterator. Unfortunately, iteration algorithms are generally completely unlike the recursive ones they mimic. They tend to be a lot trickier.# Given a list of M items and a number N, # generate all size-N subsets of M sub choose_n { my $n = pop; # Base cases return [] if $n == 0 or $n > @_; return [@_] if $n == @_; # otherwise.. my ($first, @rest) = @_; # combine $first with all N-1 combinations of @rest, # and generate all N-sized combinations of @rest my @include_combos = choose_n(@rest, $n-1); my @exclude_combos = choose_n(@rest, $n); return ( (map {[$first, @$_]} @include_combos) , @exclude_combos ); }
But they don't have to be. You can often write iterators that look like their recursive counterparts — they even include recursive calls — but they don't suffer from explosive growth. That is, they'll still take a long time to get through a billion combinations, but they'll start returning them to you right away, and they won't eat up all your memory.
The trick is to create iterators to use in place of your recursive calls, then do a little just-in-time placement of those iterator creations.
Apart from the iterator trappings, we've got essentially what we had before. Converting the map into an iterator involves some similar work, but the parallels are still pretty obvious. We exhaust the first iterator before turning to the second:sub iter_choose_n { my $n = pop; # Base cases my $once = 0; return sub {$once++ ? () : []} if $n == 0 or $n > @_; my ($first, @rest) = @_; return sub {$once++ ? () : [$first, @rest]} if $n == @_;
We now have a recursively-defined iterator that wasn't a heck of a lot more complex than our original algorithm. That's the good news. The bad news is: it's still doubly recursive, O(2^N) in space and time, and so will take a long time to start generating data. Time for a little trick. Because we don't use $exclude_iter until we've exhausted $include_iter, we can delay defining it:# otherwise.. my $include_iter = iter_choose_n(@rest, $n-1); my $exclude_iter = iter_choose_n(@rest, $n); return sub { if (my $set = $include_iter->()) { return [$first, @$set]; } else { return $exclude_iter->(); } }
Now our code is singly recursive, O(N) in space and time to generate an iterator, and that makes a big difference. Big enough that you probably won't need to go to the trouble of coming up with an O(1) truly iterative solution.# otherwise.. my $include_iter = iter_choose_n(@rest, $n-1); my $exclude_iter; return sub { if (my $set = $include_iter->()) { return [$first, @$set]; } else { $exclude_iter ||= iter_choose_n(@rest, $n); return $exclude_iter->(); } } }
Of course, if you complete the iterations, eventually you will have generated those 2^N subs, and they'll clog up your memory. You may not be concerned about that (you may not be expecting to perform all that many iterations), but if you are, you can put a little code in to free up exhausted iterators:
# otherwise.. my $include_iter = iter_choose_n(@rest, $n-1); my $exclude_iter; return sub { if ($include_iter and my $set = $include_iter->()) { return [$first, @$set]; } else { if ($include_iter) { undef $include_iter; $exclude_iter = iter_choose_n(@rest, $n); } return $exclude_iter->(); } } }
Caution: Contents may have been coded under pressure.
|
---|
Replies are listed 'Best First'. | |
---|---|
Re: Recursively-generated Iterators
by blokhead (Monsignor) on May 19, 2005 at 13:09 UTC | |
by Roy Johnson (Monsignor) on May 19, 2005 at 17:37 UTC | |
Re: Recursively-generated Iterators
by spurperl (Priest) on May 19, 2005 at 05:18 UTC | |
by Roy Johnson (Monsignor) on May 19, 2005 at 17:27 UTC | |
Re: Recursively-generated Iterators
by kelan (Deacon) on May 19, 2005 at 21:01 UTC | |
by Roy Johnson (Monsignor) on May 23, 2005 at 15:57 UTC | |
Re: Recursively-generated Iterators
by runrig (Abbot) on May 20, 2005 at 04:39 UTC | |
by Roy Johnson (Monsignor) on May 23, 2005 at 17:11 UTC | |
by runrig (Abbot) on May 23, 2005 at 18:13 UTC | |
Re: Recursively-generated Iterators
by Roy Johnson (Monsignor) on Jun 02, 2005 at 01:53 UTC |