Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Comment on

( #3333=superdoc: print w/ replies, xml ) Need Help??

Some Background

As you probably know, Perl will successfully compile nested named subroutines, but they probably won't behave as intended. Perl even tries to warn you:

use strict; use warnings; sub outer { my $foo; sub inner { $foo; } inner(); }
>perl -c test.pl Variable "$foo" will not stay shared at test.pl line 8. test.pl syntax OK

The fix is obviously to use an anonymous sub instead of a named one.

use strict; use warnings; sub outer { my $foo; my $inner = sub { $foo; }; $inner->(); }
>perl -c test.pl test.pl syntax OK

However, that fails spectacularly when recursion is introduced.

Problem 1 — Referencing the Wrong Variable

This problem is almost always easy to spot if use strict; is in effect.

use strict; use warnings; sub outer { my $inner = sub { $inner->() } }
Global symbol "$inner" requires explicit package name at test.pl line +6. test.pl had compilation errors.

Remember that a my only makes the declared symbol available in statements following the one containing the my. This can be fixed trivially by splitting the assignment into two statements.

use strict; use warnings; sub outer { my $inner; $inner = sub { $inner->() } }
>perl -c test.pl test.pl syntax OK

Problem 2 — Memory Leak

There's a subtle lesson we should have learned from the first problem: If the sub references the lexical (by capturing it), and that same lexical references the sub, then it's a cyclic structure that cannot be freed by Perl's garbage collecting mechanism.

# ReleaseTracker.pm # This "module" simply provides "o()", a # func that creates a simple object whose # sole purpose is to print "Destroyed" when # it is freed. The inner workings are not # relevant to this example. use strict; use warnings; package ReleaseTracker; BEGIN { our @EXPORT_OK = qw( o ); require Exporter; *import = \&Exporter::import; } sub new { return bless(\(my $o), $_[0]); } sub DESTROY { print("Destroyed\n"); } sub o { ReleaseTracker->new(); } 1;
use strict; use warnings; use ReleaseTracker qw( o ); sub outer { # Variables that are read by each recursive instance. my $var = shift @_; my $helper; $helper = sub { no warnings 'void'; "do something with $var and @_"; $helper->(@_); }; #$helper->(@_); } outer( o() ); END { print("On program exit:\n"); }
>perl test.pl On program exit: Destroyed <--- BAD!

$var is not being freed because the anonymous sub is not being freed.
The anonymous sub is not being freed because it both references and is referenced by $helper.

Let's illustrate:

################################################
## Before outer exits
##

   &outer
      |                       +=============+
      v              --+----->[ Reference   ]
+==============+    /  |      +=============+
[ outer's pad  ]   /   |      [ refcount: 2 ]    +=============+
+==============+  /    |      [ pointer:  ------>[ Object      ]
[ $var:    -------     |      +=============+    +=============+
[ $helper: -------     |                         [ refcount: 1 ]
+==============+  \    |                         +=============+
                   \   |
                    \  |      +=============+
                     -----+-->[ Reference   ]
                       |  |   +=============+
                       |  |   [ refcount: 2 ]    +=============+
+==============+       |  |   [ pointer:  ------>[ Helper Sub  ]
[ helper's pad ]       |  |   +=============+    +=============+
+==============+       |  |                      [ refcount: 1 ]
[ $var:    ------------+  |                      [ pad:      -----+
[ $helper: ---------------+                      +=============+  |
+==============+                                                  |
      ^                                                           |
      |                                                           |
      +-----------------------------------------------------------+


################################################
## After outer exits
##

                              +=============+
                       +----->[ Reference   ]
                       |      +=============+
    (outer still       |      [ refcount: 1 ]    +=============+
     exists, but       |      [ pointer:  ------>[ Object      ]
     it's not          |      +=============+    +=============+
     referencing       |                         [ refcount: 1 ]
     anything in       |                         +=============+
     this graph )      |
                       |      +=============+
                       |  +-->[ Reference   ]
                       |  |   +=============+
                       |  |   [ refcount: 1 ]    +=============+
+==============+       |  |   [ pointer:  ------>[ Helper Sub  ]
[ helper's pad ]       |  |   +=============+    +=============+
+==============+       |  |                      [ refcount: 1 ]
[ $var:    ------------+  |                      [ pad:      -----+
[ $helper: ---------------+                      +=============+  |
+==============+                                                  |
      ^                                                           |
      |                                                           |
      +-----------------------------------------------------------+

Nothing has a refcount of zero, so nothing can be freed.

Solution — Dynamic Scoping

The solution to both problems is the same: Don't use a lexical variable.

use strict; use warnings; use ReleaseTracker qw( o ); sub outer { # Variables that are read by each recursive instance. my $var = shift @_; local *helper = sub { no warnings 'void'; "do something with $var and @_"; helper(@_); }; #helper(@_); } outer( o() ); END { print("On program exit:\n"); }
>perl test.pl Destroyed On program exit: <-- good

Package variables aren't captured, so &helper's reference count isn't affected by the call in the inner function.

################################################
## Before outer exits
##


   &outer
      |
      v
+=================+
[ outer's pad     ]
+=================+        +=============+
[ $var: --------------+--->[ Reference   ]
+=================+   |    +=============+
                      |    [ refcount: 2 ]    +=============+
                      |    [ pointer:  ------>[ Object      ]
                      |    +=============+    +=============+
+=================+   |                       [ refcount: 1 ]
[ *helper{SCALAR} ]   |                       +=============+
+=================+   |    +=============+
[ pointer: --------------->[ Reference   ]
+=================+   |    +=============+
                      |    [ refcount: 1 ]    +=============+
                      |    [ pointer:  ------>[ Helper Sub  ]
+=================+   |    +=============+    +=============+
[ helper's pad    ]   |                       [ refcount: 1 ]
+=================+   |                       [ pad:      -----+
[ $var: ===-----------+                       +=============+  |
+=================+                                            |
      ^                                                        |
      |                                                        |
      +--------------------------------------------------------+


################################################
## After outer exits
## and local restores
## *helper{SCALAR}
##

                           +=============+
                      +--->[ Reference   ]
                      |    +=============+
    (outer still      |    [ refcount: 1 ]    +=============+
     exists, but      |    [ pointer:  ------>[ Object      ]
     it's not         |    +=============+    +=============+
     referencing      |                       [ refcount: 1 ]
     anything in      |                       +=============+
     this graph )     |
                      |    +=============+
                      |    [ Reference   ]
                      |    +=============+
                      |    [ refcount: 0 ]    +=============+
+=================+   |    [ pointer:  ------>[ Sub         ]
[ helper's pad    ]   |    +=============+    +=============+
+=================+   |                       [ refcount: 1 ]
[ $var: --------------+                       [ pad:      -----+
+=================+                           +=============+  |
      ^                                                        |
      |                                                        |
      +--------------------------------------------------------+

There is no cycle, so everything will be freed in turn, starting with the reference with a refcount of zero.

Alternative Solutions

I won't delve into these, so feel free to provide links which discuss these in more details. For each alternative solution, I'll post the equivalent to the solution I've already presented.

sub outer { ... local *helper = sub { ... helper(...); ... }; helper(@_); }

Y Combinator

ambrus pointed out that Y Combinator can also achieve this goal. This is the most worthwhile alternative.

sub outer { ... Y( sub { my $helper = shift @_; sub { ... $helper->(...); ... } } )->(@_); }

While it's a great tool to completely anonymize a recursive function, and while it might even be required in functional languages, it adds overhead in a situation where reducing overhead is important, and I think it's unnecessarily complex to solve the problem of nesting functions in Perl.

Weak Reference

One could weaken the reference to the inner function using Scalar::Util::weaken.

sub outer { ... my $helper; $helper = sub { ... $helper->(...); ... }; weaken($helper); $helper->(@_); }

However, the Weak Reference solution is much more complex than the Dynamic Scoping solution and has no advantage that I can see.

Summary

Using local *helper = sub {}; over my $helper = sub {}; not only provides a cleaner calling syntax, it can be used for recursive functions without accidentally referencing the wrong variable or causing a memory leak.


In reply to Creating Nested Functions by ikegami

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • Outside of code tags, you may need to use entities for some characters:
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others rifling through the Monastery: (7)
    As of 2014-12-28 14:25 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?

      Is guessing a good strategy for surviving in the IT business?





      Results (181 votes), past polls