Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Solving the SUPER problem in Mixins with String Eval

by simonm (Vicar)
on Oct 11, 2004 at 00:43 UTC ( #398061=perlmeditation: print w/ replies, xml ) Need Help??

I've recently stumbled upon a technique for creating mixin classes which I thought might be worthy of some discussion here.

Motivation
When creating an object class that will be used by different people for different purposes, I find myself drawn to solutions in which a minimal base class provides the shared behavior they all need, and a collection of subclasses provides layers of additional functionality.

As an example, I've recently been refactoring Text::MicroMason to allow dynamic subclassing. The Text::MicroMason::Base class provides methods to convert templates from HTML::Mason's syntax to runnable Perl code. Other classes extend that behavior in different ways, such as running code in a Safe compartment or adding support for the "|h" and "|u" filter flags.

A naive implementation of this might use a subclass for each behaviour, and look like the following:

		      +---------+
		      |   Base  |
		      +---------+
			   |
	+------------------+------------------+
	v                  v                  v
   +---------+        +---------+        +---------+
   |  Cache  |        |  Filter |        |   Safe  | @ISA=qw(Base)
   +---------+        +---------+        +---------+

The well-known problem with this implementation appears when you want to combine several features:

		      +---------+
		      |   Base  |
		      +---------+
			   |
	+------------------+------------------+
	v                  v                  v
   +---------+        +---------+        +---------+
   |  Cache  |        |  Filter |        |   Safe  | @ISA=qw(Base)
   +---------+        +---------+        +---------+
			   |                  |
			   +---------+--------+
				     v
			      +-------------+
			      | Safe_Filter | @ISA=qw(Filter Safe);
			      +-------------+

This is the dreaded "diamond inheritance" problem: if Base provides a compile() method, which Filter and Safe each override to perform additional actions around a call to SUPER::compile, how can we ensure they are all called in sequence?

Diamond inheritance is avoided if we use mixin classes, which don't themselves inherit from the base class:

   +---------+        +---------+        +---------+
   |   Base  |        |  Filter |        |   Safe  | 
   +---------+        +---------+        +---------+
	|                  |                  |
	+------------------+------------------+
	                   v
		    +-------------+
		    | Safe_Filter | @ISA=qw(Filter Safe Base);
		    +-------------+

However, in this condition our mixin classes can't call SUPER methods at all!

Combining Mixins with Run-time Evals
My idea was to combine the mixins approach with a judicious spot of run-time string evals to automatically generate the inheritance tree that was needed.

In this implementation, the mixin classes can both define some methods normally, and also provide some method definitions in string form, as part of a @MIXIN array. When a request for a class with a particular combination of features is received, a series of new classes are created on the fly.

  # User calls Text::MicroMason->new( '-Filter', '-Safe' )

  +---------+            +---------+            +---------+
  |   Base  |            |  Filter |            |   Safe  | 
  +---------+            +---------+            +---------+
	|                      |                      |
	+-----------+----------+                      |
		    v                                 |
	     +-------------+ eval join "\n",          |
	     | Filter_Base |  '@ISA=qw(Filter Base);',|
	     +-------------+   @Filter::MIXIN;        |
		    |                                 |
		    +-----------+---------------------+
				v
			+----------------+ eval join "\n",
			|Safe_Filter_Base|  '@ISA=qw(Safe Filter_Base);',
			+----------------+   @Safe::MIXIN;

Although string evals are notoriously slow, we only need to do this once for each new combination of mixins, so the run-time impact is minimal.

If the mixin classes define their compile() methods in @MIXIN, calling the compile() method on a Safe_Filter_Base object will now properly follow SUPER calls up the inheritance chain. And of course calls to the other, non-SUPER-calling methods will continue to follow the hierarchy and perform as expected.

A typical definition for a method that would be defined in the @MIXIN array might look like the following example. The BEGIN line snags the file name and line so that error messages are reported in the right place, and captures the text of the following subroutine with a here-doc.

package Text::MicroMason::Benchmarking; use Time::HiRes 'time'; BEGIN { push @MIXIN, "#line ".__LINE__.' "'.__FILE__.'"', "", <<'/' +} sub compile { my $self = shift; my $time = time(); my $result = $self->SUPER::compile(@_); $self->show_results( $time, time() ); return $result; } / sub show_results { my ( $self, $start, $end ) = @_; warn "Compilation took " . ( $start - $stop ) . " seconds"; } 1;

The order in which mixins are stacked is significant, so the caller does need to have some understanding of how their behaviors interact. For example, you'd typically want to ensure that the Benchmarking mixin was the first in the chain, so that it could time everything later in the sequence.

Related Techniques
None of this would be necessary if I used the NEXT module, which differs from SUPER in allowing method invocations to back-track through the inheritance tree and explore other branches.

There are some good non-inheritance approaches to this class of problem, particularly the use of decorators and delegation; for example the Benchmarking object could hold a reference to the base object, and the base object could accept an optional reference to a separate Lexer object. While I'm a big fan of composition, I also like the way this mixin technique works and think there may be cases for which it's a better fit.

In the long term, all of this will presumably be changing under Perl 6; perhaps the roles mechanism will provide a unified way of combining behaviors, but I haven't fully wrapped my head around its day-to-day implications.

Further Work
I think that this is a generally applicable technique, but I haven't yet used it in enough cases to have a sense of how to abstract the solution into a stand-alone CPAN module. Module import() time seems the right place for the inheritance and symbol table munging.

I'd particularly like to make the method definition more transparent, so that you didn't have to explicitly fill @MIXIN, but I'm wary of using source filters. Is there a way to avoid string evals here by rebinding a clone of a subroutine to a new package, so that SUPER resolution will start from that other class? (I tried using Exporter to import the methods into the target package, but SUPER still seemed to start searching from the package in which they were compiled.)

If you'd like to look at some working examples of this kind of mixin code, check out Text::MicroMason and DBIx::SQLEngine::Record::Class.

Feedback welcome.

Janitored by Arunbear - added readmore tags

Comment on Solving the SUPER problem in Mixins with String Eval
Download Code
Re: Solving the SUPER problem in Mixins with String Eval
by hv (Parson) on Oct 11, 2004 at 10:50 UTC
    Is there a way to avoid string evals here by rebinding a clone of a subroutine to a new package, so that SUPER resolution will start from that other class?

    I'm not aware of any way of retargetting SUPER resolution. I have some similar but different needs in my work application, and I solve them by injection: I construct a class specific to the caller which is injected into the caller's @ISA, and use string eval to recompile each of the methods supplied by the mixin into that class.

    So in one module I might have:

    package NVC::DB::User; our @ISA = qw/ NVC::DB /; use NVC::DB::attr qw/ cache /; ...
    and NVC::DB::attr will create a class NVC::DB::cache::User into which to inject the extra methods, and set:
    @NVC::DB::cache::User::ISA = @NVC::DB::User::ISA; @NVC::DB::User::ISA = qw/ NVC::DB::cache::User /;

    Since I'm using the string eval anyway, I take advantage of it to construct mixin methods specific to the adopting class rather than creating catch-all methods that do runtime tests to determine the correct behaviour for the class. So, for example, the cache mixin uses different code depending whether the adopting class has a single key field or multiple key fields.

    Since this is all transparent to the adopter, I can turn caching on or off by adding or removing use NVC::DB::attr qw/ cache /; in the class definition.

    Hugo

      I'm not aware of any way of retargetting SUPER resolution.

      When you call $obj->SUPER::foo you're actually calling the function SUPER::foo, which is caught by something like an AUTOLOAD. NEXT works the same way, except that it actually IS an AUTOLOAD that does the catching.

      Therefore if you wish to retarget SUPER::foo, you can write a SUPER::foo function and it will happen. But you'll catch everyone's SUPER::foo, so program carefully...

Re: Solving the SUPER problem in Mixins with String Eval
by stvn (Monsignor) on Oct 11, 2004 at 15:36 UTC

    Very nice meditation and very interesting technique (I took a look at some of the "guts" of it here as well). However I must say that it seems like a lot of overhead in single inheritance situations.

    This is the dreaded "diamond inheritance" problem: if Base provides a compile() method, which Filter and Safe each override to perform additional actions around a call to SUPER::compile, how can we ensure they are all called in sequence?

    In my experience, it is not usually just an issue of FilterSafe calling them in the right sequence, but doing what you say Filter and Safe do and performing additional actions around calls to the superclass's compile method. So while many times I would like to be able to just use something like NEXT, I find that I must instead do things like $self->Filter::compile() and $self->Safe::compile() instead. And to be honest this no longer bothers me, and I have now learned to see it as an elegant way to specifically direct method dispatch in the presences of multiple inheritance.

    However, your medition got my mind moving and I decided to see if I couldn't create a simple NEXT-ish type module which would dispatch to all the superclasses in the same order as the modules were inherited, and without some of the odd craziness which NEXT seems to require (in the examples all calls go through NEXT:: even in the base classes and single inheritance classes).

    I am not sure if this would solve the specific problem you have though, but it does solve the "diamond inheritance/called in sequence" issue you mention. Anyway, back to work, thanks for the mental jumpstart, very nice meditaion.

    -stvn
Re: Solving the SUPER problem in Mixins with String Eval
by SpanishInquisition (Pilgrim) on Oct 11, 2004 at 17:30 UTC
    You appear to be using the "inheritance is my hammer" ideology... Lemme try to explain. Maybe it's just me, but I seem to have managed to unlearn much of my OO schooling due to seeing industry abuses of it, and have rediscovered how to make OO fun again.

    I see over-reliance on inheritance as one of the major pitfalls of OO design. When you see something complicated, the right thing to do is not to hack around it to make a more complicated solution, but to step back, look at the design, and clean it up. In this case, inheritance is your hammer, but the solution appears to be composition and chaining objects together.

    If you are making a FooBarThing, don't let it extend FooThing and BarThing, or mixin Foo or Bar and subclass Thing, it doesn't have to be so complicated.

    Have a MetaThing that takes any array of Things, and (though some combination functions) runs them together. If Things were Filters, a FooBarThing might run the filter results of Foo into a Bar, for example. This can even be totally dynamic, as in MetaThing(new FooThing,new BarThing).

    If FooThing and BarThing are not sufficiently alike to have a common superclass, we have a problem, and we appear to be going down the mixin path. But I have found that if you make your objects real enough (nothing like Base without knowing what a Base really IS), you don't go down the mixin path.

    I can go on further, but in case this doesn't make sense I'll wait and see if anyone understands what I've just said. The basic idea is "inheritance is only one tool in OO; and it is overused -- we can blame some of this on bad schooling from Java -- KISS still applies". If the mighty hammer of composition/encapsulation and polymorphism were used at an equal level as inheritance to solve design problems, much of the world's code would be a lot simpler and more maintainable.

    As we go to further and further extents to make OO feel right, we lose the grasp of what OO is supposed to be. Keep it simple, design such that these situations do not occur, and when they do (and they will), find simple solutions out without symbol table manipulation and evals and such -- it is better to keep things clean and basic.

    Anyhow, by no means am I an expert in my theory, and it's not the only way, but I do find OO much less distasteful when I start to temper the inheritance-hierachy thoughts that are so common these days, and start to put order back into designs. OO design that can't be articulated in simple diagrams can't succeed. The entire black box concept can't devolve into the "great ball of mud" designs java programs face. Anyhow, this has little to do with simon...but it's an interesting thing to watch. Using inheritance as the proverbial hammer is a gateway drug to overcomplicated interfaces. Mix in sic some functional programming, and remember what procedural programming taught you about simplicity being the only way to maintaince. And by no means build a shoehorn. Ever. (Unless you run a cobbler shop)

      I see over-reliance on inheritance as one of the major pitfalls of OO design

      Personally, I think that a better more general was to say that is, "The mis-use of inheritance is one of the major pitfalls of OO Design". Over-reliance is only one of the problems out there, under-reliance can be just as bad (when people don't understand when to use it, rather then just not knowing when not to use it).

      I agree with a lot of what you are saying, there is far to much inheritance shoehorning going on these days (although that is not really what simonm is doing, you might want to look closer at the implementation in Text::MicroMason (make sure you are looking at the latest developer release)). I have always felt that inheritance is best when it is employed in seperate clusters of shallow heirarchies which (usually) are not related to one another (a.k.a. highly decoupled).

      ... I can go on further, but in case this doesn't make sense I'll wait and see if anyone understands what I've just said.

      What you just described is actually an excellent case for a specific style of inheritance. Interface Polymorphism is IMO of the great underused and misunderstood aspects of OO programming.

      To do what you describe (have a MetaThing which takes an array of Things and "runs" each of them in turn) it is usually a good idea to have a common base which all Things are derived from. If for no other reason then for MetaThing to be able to indentify each Thing as a Thing. The knee-jerk approach is to make Thing::Base and have all Things inherit from it. However, this approach limits you in several ways since all your Things must inherit from Thing::Base.

      However, a better solution to this all is to create a Thing interface. Then all MetaThing needs to know is that your Thing conforms to the Thing interface, everything else is irrelvant to MetaThing. The idea really is that you try not to program "to the class", but instead "program to the interface". I have found that thinking this way tends to lead to highly reusable code and very clean designs.

      -stvn
        Yep, we are saying the same thing -- perhaps in different accents, but basically the same thing. At least the way you interpret the Things is the way I meant to explain them. As far as interfaces go, I'm not sure if I agree with the uber-dynmaticity of it all, but that's the way classes in Ruby work...you test to see if something responds to a particular function, rather than seeing if an object is_a particular thing. Again, I'm not sure I totally buy it, but it's an interesting idea.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://398061]
Approved by rlb3
Front-paged by Arunbear
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (9)
As of 2014-12-21 14:53 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (106 votes), past polls