Re: Eliminating Recursive Tree Walking by using MJD-style Infinite Streams?

[ikegami]

Depending on your data, you might save a lot of memory by changing from

$paths->[0][0][0] = [
   [ 3, 0, 0 ],
];
  
$paths->[0][1][0] = [
   [ 0, 2, 0 ],
];

...

to

$paths->{0,0,0} = [
   join($;, 3,0,0),
];
  
$paths->{0,1,0} = [
   join($;, 0,2,0),
];

...

For starters, you have way fewer values, and hashes are better for sparse data. You could even pack the data down if you know the range of the numbers.

sub to_key {
   my ($x, $y, $s) = @_;
   return pack 'L', ($x-MIN_X) * (MAX_Y-MIN_Y) * (MAX_S-MIN_S)
                  + ($y-MIN_Y)                 * (MAX_S-MIN_S)
                  + ($s-MIN_S)
}

sub fr_key {
   my $key = unpack 'L', $_[0];
   return (
      int( $key / (MAX_Y-MIN_Y) / (MAX_S-MIN_S) )                 + MI
+N_X,
      int( $key                 / (MAX_S-MIN_S) ) % (MAX_Y-MIN_Y) + MI
+N_Y,
      int( $key                                 ) % (MAX_S-MIN_S) + MI
+N_S,
   );
}

$paths->{to_key(0,0,0)} = join '',
   to_key(3,0,0),
);
  
$paths->{to_key(0,1,0)} = join '',
   to_key(0,2,0),
);

...

fr_key( substr($branches, $i*4, 4) ) would be used instead of @{ $branches->[$i] }.

---

Aside from that, moving to an iterator (as you suggested) would also help. Your current function generates *all* paths before returning. Instead, it could return a path as soon as it finds one.

---

But before you do anything, are you sure the deep recursion is simply deep (not a problem, except you might run out of memory) and not infinite?

[jryan]

Its certainly not infinite recursion. Its just deep. Which is fine, I don't care about warnings. On some cases it hits deep recursion and ends up resolving just fine. Its just that on other inputs it first hits deep recursion and THEN hits "Out of memory!".