Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

Re: Runtime introspection: What good is it?

by sgifford (Prior)
on Jul 09, 2008 at 04:06 UTC ( #696377=note: print w/ replies, xml ) Need Help??


in reply to Runtime introspection: What good is it?

Here are some of the things I've used run-time reflection of some kind for, in various languages:

  • When how to perform an operation depends on the type of more than one thing. For example, if you have a Shape class and want to implement Shape::intersection to compute the intersection of two shapes. Dispatching based on one of the shapes is easy; $shape1->intersection($shape2) will find the type of $shape1 and dispatch appropriately. But unless the other shape's type is known at compile-time, there will need to be some kind of run-time type information available to know how to compute the intersection. If $shape2 is coming from a database or the network, its type certainly won't be available at compile-time.
  • Loading things at run-time. For example, I wrote a streaming database system in Java that started by reading a configuration file, then loaded the classes named there. It was best to look at the class as soon as it was loaded and make sure it was a type I could deal with, so more information was available for error reporting.
  • Automatically providing information for things like GUI editors and SOAP interface generators. It is convenient for these systems to be able to walk a class's methods to see what it can do, to make that functionality available over a network connection or a GUI.
  • Displaying information. I have a program that manages several different types of objects. I just added on a GUI displayer, and the easiest way to do that was to write a little display module, and have it say "if this is type 1, display it this way; if it's type 2, display it this way; etc...". It could have been done through subclassing, but it would have been much more complicated.
  • Working around quirky behavior. I may know that a particular class will handle a particular situation incorrectly, so I can detect that class at runtime and try to work around the problem. For example, if you know a particular class has trouble with unicode, maybe you strip it out before sending it to that class.


Comment on Re: Runtime introspection: What good is it?
Select or Download Code
Re^2: Runtime introspection: What good is it?
by BrowserUk (Pope) on Jul 09, 2008 at 19:40 UTC

    This got rather long. Testimony to the thoughtfulness of your use cases; thank you again.

    Here are some of the things I've used run-time reflection of some kind for, in various languages:
    • When how to perform an operation depends on the type of more than one thing. For example, if you have a Shape class and want to implement Shape::intersection to compute the intersection of two shapes. Dispatching based on one of the shapes is easy; $shape1->intersection($shape2) will find the type of $shape1 and dispatch appropriately. But unless the other shape's type is known at compile-time, there will need to be some kind of run-time type information available to know how to compute the intersection. If $shape2 is coming from a database or the network, its type certainly won't be available at compile-time.

      (Starting with the bit I've emboldened)Sorry, but the second half of that sentence is obscuring the situation.

      You will have to know what type of shape is is, in order to instantiate the object. True regardless of whether you have:

      • A generic Shape class that (say) represents all shapes as lists of vertices. In which case you would call a single constructor for all shapes coming from the DB or network will be of type Shape and that will be known at compile time.
      • Specialised (sub) classes for each type of shape (Rect Circle Polygon). In this case, something within the data input will identify what shape this set of data represents, and you will then you will call the particular constructor for that shape. And all of those constructors will be known at compile time.

      This is not introspection. Because you cannot introspect an object until it exists. And you cannot construct it into existence if you need to introspect it to do so.

      This is simply data-driven code. You read the data, inspect a field with that data, and then dispatch to a constructor based upon what you see there. This can be done in good ol' introspection-less C something like:

      char * buffer = malloc( ... ); read( source, buffer ); int type = buffer[ 0 ]; void *o; switch( type ) { case RECT: o = makeRectFromString( buffer ); break; case ELLIPSE: o = makeEllipseFromString( buffer ); break; ... }

      And if you can do it this way in C, you can do it in any other language, including those that support reflection.

      For your intersection problem, any language that supports method overloading, C++, Java etc., will allow you to code methods within your Rect subclass with signatures of:

      class Rect; bool intersect( Rect* ); bool intersect( Ellipse* ); bool intersect( Polygon* ); ...

      So that invoking someRect->intersect( someShape ); will get to the right code without introspection.

      In Perl, which doesn't support method overloading, you would have to dispatch internally to the (single)intersect method, but that's a Perl OO-model limitation.


    • Loading things at run-time. For example, I wrote a streaming database system in Java that started by reading a configuration file, then loaded the classes named there. It was best to look at the class as soon as it was loaded and make sure it was a type I could deal with, so more information was available for error reporting.

      This is the 'plug-in' scenario. Loading a class at run-time from a filename read at run-time. This part is data-driven. It cannot be reflection, since there is nothing in memory upon which to reflect.

      For the "make sure it was a type I could deal with" part of the equation, if all plug-ins are derived from a base class, then the only check required is to verify that the class loaded is derived from that base class.

      This can be done through Java.Lang.Class or Java.Beans.Introspector. But, as I found in several sources whilst refreshing my latent memories of these, there are costs:

      The Costs of Usage

      Reflection and Introspection are powerful tools that contribute to the flexibility provided by the Java language. However, these APIs should be used only as needed and after taking into account the costs associated with their usage:

      • Reflection and Introspection method calls have a substantial performance overhead.
      • Using reflection makes the code much more complex and harder to understand than using direct method calls.
      • Errors in method invocation are discovered at runtime instead of being caught by the compiler.
      • The code becomes type-unsafe.

      What's the alternative? try{ ... } catch{ ... }

      Once the class is loaded, try instantiating a (minimal) instance, and exercising the required methods. Catch and report any errors. You can even check that the loaded class methods return sensible values--And that's something that no amount of reflection can do. All your validation is performed immediately after loading.

      The rest of your main application can be written against the plug-in base class. As you can pass or use an instance of a derived class anywhere you can use an instance of its base class, your main application will written in terms of direct method calls, and be compile-time type-checked. No further need for run-time type checks or try/catching.

      You simply define all your main application method calls as taking instances of the base class, and pass instances of the run-time loaded derived plug-in class.

      The advantages are all the opposites of the costs listed above. And, as you do not need to use reflection, you can use this technique in any language that supports runtime loading of classes and exception handling. Eg. Perl 5.


    • Working around quirky behavior. I may know that a particular class will handle a particular situation incorrectly, so I can detect that class at runtime and try to work around the problem. For example, if you know a particular class has trouble with unicode, maybe you strip it out before sending it to that class.

      I think what you are saying here is that you can use reflection to construct a workaround to the quirk, not detect the need for it. If you know enough to write code to use the reflection APIs, you know enough to write a non-reflective solution--if that is possible. The question then becomes, is a non-reflective solution possible, and that will depend very much on what the quirk is, and what language your using.

      But the classic solution to this is to construct a subclass that inherits from the quirky class and override the troublesome methods. For your unicode-unaware example, you convert from unicode on the entry to the subclass methods, call the superclass method, and convert any returned strings back to unicode on the way out.

    • Displaying information. I have a program that manages several different types of objects. I just added on a GUI displayer, and the easiest way to do that was to write a little display module, and have it say "if this is type 1, display it this way; if it's type 2, display it this way; etc...". It could have been done through sub-classing, but it would have been much more complicated.
    • Automatically providing information for things like GUI editors and SOAP interface generators. It is convenient for these systems to be able to walk a class's methods to see what it can do, to make that functionality available over a network connection or a GUI.

      If you code each method with a (Java-style) toString() method, this problem is 'solved'.

      Yes, I know that doesn't solve the problem for third party classes that either fail to define a toString() method, or define one and return "Verboten!Keep your nose out sucker!", or worse :)

      More seriously, if I supply symbol files for libraries to my Symbolic debugger, then if can decode and display the contents of structs and the like. Similarly, the built-in GUI editor in the only IDE I ever spent any serious time with (PWB), utilised those same symbol files plus other compiler generated files to do some pretty remarkable things including single stepping the code backward as well as forward.

      But I will admit that these are both good use cases for reflection. The questions it leaves me with are:

      1. Is it necessary or desirable to load up every class, of every application, with all the overhead that reflection entails, in order to meet these use cases?
      2. Or would it be better to follow the PWB practice of placing this information in an ancillary file during compilation, and only loading it for those (few) use cases that need it?

        Or in the case of a dynamic language where compile-time is the early stages of run-time, have a run-time switch on the interpreter that enables reflection? Or perhaps a per-class or per-module pragma that enables it?

    (My) conclusions

    1. I have no doubt that all of these use cases could be met using a language (like C) that has no introspection capabilities, if the need was strong enough. It might entail using compiler generated ancillary files, or in effect, constructing a limited reflection capability.

    2. As I think I've shown, three of the five above can not only be met using standard OO mechanisms, I would say that those three would be better met that way.

    3. The other two, (grouped together and shown last), are strong use cases for reflection, where that capability is available.

      Though whether they are common enough to warrent the inclusion of introspection in a language is open to debate. If the reflection is done, as in Java, by (as I understand it), pulling apart the bytecode at run-time, the costs when it is not used are negligable.

      But in languages or frameworks where introspection must be support through the retention of compile-time parsing tables or the construction of code tree decorations, the costs in terms of space and time can be significant enough that they should only be done on demand, through the use of compile-time switches or pragmas, not by default.

    4. In general, I think that just as inheritance was overused and abused in the poineering days OO, and still is by novice users, so introspection is equally easily abused. And 5 or 10 years from now, it will probably have gained as bad a rep. when people step back from its rising popularity on the basis of RoR and resurgent interest in Objective-C.

      Like any technology, used sparingly with understanding of the costs, it will serve some use cases in ways that nothing else can.

      But if it is overused, just because it can, to solve at run-time use cases that are better served by compile-time solutions, I see it suffering a backlash as the mainenance costs become evident.

    I hope that if anyone read this far, they will find some of this as useful as I have.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      Hi BrowserUK,

      Glad to be able to make you think a bit! Let me try to respond to a few of your comments. But first, we should be clearer about our definitions of reflection. I'm including in my definition finding the type of an object (using Java.Lang.Class in Java or typeof in C++ or ref in Perl), and testing whether an object inherits from another object (using instanceof in Java or dynamic_cast in C++ or isa in Perl. If you don't consider these reflection, and are only thinking of getting the methods and member variables of a class, some of my examples don't really use reflection.

      You will have to know what type of shape is is, in order to instantiate the object
      Right, this part doesn't use reflection.
      For your intersection problem, any language that supports method overloading, C++, Java etc., will allow you to code methods within your Rect subclass with signatures of:
      class Rect; bool intersect( Rect* ); bool intersect( Ellipse* ); bool +intersect( Polygon* ); ...
      So that invoking someRect->intersect( someShape ); will get to the right code without introspection.
      This is called "dynamic dispatch" IIRC and the behavior I would like and expect, but unfortunately not the behavior exhibited by either Java or C++. Both determine which overload of a function/method to call at compile-time, and if all you know about the object is that it's a Shape* it will always call the overload for that type. For example, in C++:
      #include <iostream> using namespace std; class Shape { public: virtual ~Shape(){}; }; class Polygon : public Shape { }; class Circle : public Shape { }; void ShowType(Shape *obj) { cout << "Shape" << endl; } void ShowType(Polygon *obj) { cout << "Polygon" << endl; } void ShowType(Circle *obj) { cout << "Circle" << endl; } int main() { Shape *shape; shape = new Polygon(); ShowType(shape); shape = new Circle(); ShowType(shape); }
      outputs:
      Shape Shape

      To make it work, we have to use runtime reflection:

      void ShowType(Shape *obj) { if (dynamic_cast<Polygon*>(obj)) ShowType(dynamic_cast<Polygon*>(obj)); else if (dynamic_cast<Circle*>(obj)) ShowType(dynamic_cast<Circle*>(obj)); else cout << "Shape" << endl; }
      outputs:
      Polygon Circle

      This is why the distinction between knowing the type at compile-time versus runtime is important; if the compiler knows the type it can call the correct overload, otherwise it will not.


      For the "make sure it was a type I could deal with" part of the equation, if all plug-ins are derived from a base class, then the only check required is to verify that the class loaded is derived from that base class.
      Right, but IIRC the class loading code returns a Class and you have to use runtime type checking to determine if it is the right sort of class.
      Once the class is loaded, try instantiating a (minimal) instance, and exercising the required methods. Catch and report any errors. You can even check that the loaded class methods return sensible values--And that's something that no amount of reflection can do. All your validation is performed immediately after loading.
      While this is possible, it is error-prone, and violates the concept of putting things in exactly one place. If I add a new method to my class, I have to remember to go add a check for that method to all places where it is loaded dynamically. If it is loaded dynamically from 3rd party code, I have to notify those parties to check for this new method. The maintenance cost is much higher than the runtime cost of doing this check, IMHO.

      I think what you are saying here is that you can use reflection to construct a workaround to the quirk, not detect the need for it
      I'm actually simply saying to detect a need for it, by looking at the type of the object.
      the classic solution to this is to construct a subclass that inherits from the quirky class and override the troublesome methods
      Easier said than done if your code is a library being used by other programs who are creating the object in question and passing it in. All users of your code would have to switch to your subclass, then switch back when the bug in the original code is fixed, which is a maintenance nightmare.

      As far as the cost of keeping type information and reflection information around, I'm not quite sure what the cost is in different languages. I think it's quite low for C++ RTTI, and I think Perl has to keep that information around anyways, so it's also quite low. But I haven't really seen reflection overused; it seems to be just inconvenient enough that it doesn't get used unless there is a genuine need.

        Have you heard of or used "late binding"? Try it this way

        #include <iostream.h> class Shape { public: virtual ~Shape(){}; public: virtual void ShowType() { cout << "Shape" << endl; } }; class Polygon : public Shape { public: virtual void ShowType() { cout << "Polygon" << endl; } }; class Circle : public Shape { public: virtual void ShowType() { cout << "Circle" << endl; } }; int main() { Circle circle; Polygon polygon; Shape *shape1 = &circle; shape1->ShowType(); Shape *shape2 = &polygon; shape2->ShowType(); }

        Outputs:

        c:\test>696596 Circle Polygon

        This is essentially the same as "re-blessing" a blessed-reference in Perl 5.


        If I add a new method to my class, I have to remember to go add a check for that method to all places where it is loaded dynamically.

        If you add a new method, you are going to have to modify any program that uses the class anyway, otherwise it will never be called. Reflection will tell you that it's there, but it won't tell how to call it, when to call it, what it will do, or why.

        So, as you 've got to go in there and modify any program that uses the modified base class anyway, you might as well update the run-time sanity-checking code to test if the objects you are instantiating have been upgraded with that new method also.

        And checking that it appears to do the right thing at run-time is just good insurance. Think of it as adding another test to your test suite. Except that it is one that protects your code by checking that other people (whomever writes the plug-ins derived from your plug-in base class), have done theirs.


        Easier said than done if your code is a library being used by other programs who are creating the object in question and passing it in.

        Okay. Your scenario is that you are providing a library that accepts instances of the errent class, that have been instantiated by code that you did not write.

        So, you write your library in terms of the subclass of that errent class as I described above. Whenever your code gets an instance of that errent class, you use the late binding trick to "re-bless" it on the way in as as above. And then reverse the process (assignment) on the way out.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://696377]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (16)
As of 2014-09-22 20:44 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (200 votes), past polls