Do you know where your variables are? | |
PerlMonks |
RFC Is this readable?by BerntB (Deacon) |
on Oct 16, 2005 at 12:56 UTC ( [id://500554]=perlmeditation: print w/replies, xml ) | Need Help?? |
My self confidence for writing readable documentation isn't the best. This document took longer and was more pain than I'm willing to admit. :-)
This document is maybe 75% done. I'd like to know if it is understandable? How should I reorganize it? I'm not asking for detailed criticism yet (but would appreciate it), but more high level. If you hate the idea or my implementation, say so. Politely, please. :-) In addition to this, there is an Introduction (I put it here: BerntB's scratchpad) and pod files documenting the method calls. You might have to read the intro to understand this. (If anyone is curious and willing to spend time analyzing this and give feedback, I can email what I have. There is some work left for an alpha at CPAN. I'll GPL it, I think, to help get examples published.) Oh, is Tree::Model a good name?
IntroductionThis is about how to write definition files (Declarations) for the Tree::Model module and gives an overview of the programming API. Read the Introduction document before looking at this, so you understand what Tree::Model does. After this document, you should read the programming APIs and the examples. Start with the pod/documentation for top.pm and then sysOwner.pm.
General advice for building modelsFirst, do you really need this module? A XML editor or a spreadsheet is probably the right answer in 90++% of the cases this might be applied. This is optimized for quite complex data descriptions with many calculations and complex rules with many exceptions. Be aware of that it is quite expensive to run this design program. Especially if you have a standard mod_perl environment where data is read in whenever anything will be done to them. A RPC interface would be quite easy to do but isn't implemented yet. Define your data model in phases. Start with just the objects, some start Instances and very general inheritance (allowbelow) rules. Generate data and experiment. Then think a little about how to show the user interface -- even if you are going to have another program using this, you need to test graphically so check the html generators. Think about how the additions/deletions of the structure would be managed. What are the rules and where are the data needed to decide if an addition or a deletion is allowed? How should data be generated? Just as XML or as an object tree? So, at this point you have played with the object structure, knows that it can be presented without too much pain and have planned the code for building the structure. You should be quite certain you haven't painted yourself into a corner that needs hard solutions to get out of. Next is implementation. User interface, control and calculations.
DecoratorsDon't overuse Decorators and don't give them too much (if any) state! They are there to extract parts of very complex code and special cases that might be applicable at certain times and not others.
Excuses :-)It is scary to let so many pages code out for public viewing, so here comes my excuses. :-) This module was written to solve a hard modelling problem and grew organically depending on needs. I was quite surprised when I realized what I was creating and skipped my original idea in favor of making a general utility out of this. If I redid this, I'd do an extra level of verification for XML. I had programmed quite a bit of Perl before starting this project, but had always tried to keep it close to ``normal'' programming languages so non-Perl people at work could keep up. I have started to use more Perl idioms as time went. Also, it was a hobby so I have written parts just for fun (i.e. config files instead of just declaring data structures in files). This should have been split into a number of modules according to functionality (object orientation, tree handling, etc). Mostly because it would be easier to understand but also because it might have been neat modules for CPAN.
ConceptThese are important concepts in the Tree::Model module and how the terms are used. The order is not alphabetical, but more in increasing level of hmm.. how ``esoterical'' they are.
Programming the High Level API (UI-API)The calls of the UI-API are documented in the pod of sysOwner.pm, see the CGI program for examples. This is an overview. The ``UI-API'' is used for communicating with a Tree::Model data model, probably by a program implementing a user interface. The ``normal'' programming API is used by objects to implement the model -- do calculations and build rules/logic to implement the high level operations. The UI-API talks to that model.
Program structureThe central part of the UI-API is the Describe call which returns the Tree:Model data formatted to be shown to the user -- with information about actions on the data which might be done. The possible actions are deleting/adding an object, updating a value and answering a specific query from an object. The program presents the data and the possible actions in a user interface and asks for a choice. The selected command is sent back to the module. After that, a new Describe call is issued and the new data is presented, ready for the next user choice. This Describe call is expensive and is also used often -- a ``refresh'' must be done at all changes. Since the data model has code embedded, changes in one place might influence objects far away in the hierarchy (this update is expensive, too). The point is that the program structure of a user interface is simple, really. It doesn't need to know much more than how to present the data, take the commands and send them back to the user. There are examples of all this. The complex part is how to present data in a practical way. The problem with this simplicity of design is that modelling problems with data are hard to test from only the UI-API. At a minimum, a data dump from the ``real'' data model needs to be done and analyzed. Test files are recommended.
Object IDs and NamesWhen a data structure representing the data model is read with the UI-GUI, all objects have unique names and ID-strings. Calls in the UI-GUI will use these ID strings as references to objects in the data model. The unique names are suited to showing to the user.
Create a new modelTo find the possible ways to start a data model, use the command list_templates without parameters. It returns an array with hashes, where every hash represent an alternative. The hash has keys name and description. Use the new_top command to create one of those models as a new object hierarchy. It returns the ID of the top object, which is used to generate a description. (Right now, only one is supported in memory for a system. This will probably be lifted in the future.) To see what top-level objects that exist, use the command list_tops, no parameters.
Load/StoreThe high level API has the commands savedata and getdata. Note that they assume that a configuration directory should have been set up with SetDataDir before calling. They store the data based on the user name and data name. (Done like this to be easy to store in a database instead.)
Get data descriptionTo read data, use the describe command with a parameter for what ID to read from. The returned data is a hierarchical definition of the system. Data invisible to the user isn't returned (see the noread attribute for variables). Object descriptions are in the form of a hash. Sub-objects are in an array (order is relevant). Attribute and variable information is added. Here are also a representation for the Query interface (specific commands to objects, optionally with parameters). See the pod/documentation for the data format.
Create a new data objectCreating an object beneath another is done in two phases in a GUI-client; first the command okbelow is given and then a list of classes for allowed objects is generated. The user choose what to try to add. Then an object (or Instance) is added below the object, using the UI-API's command addto. The reason this is done in two phases is that it might be expensive to generate an exact list of what is allowed below another object -- and the list might change dynamically with every command. Note that this is the only place you see Instances in the UI-GUI. They are a handy way of specifying alternative names and attributes for a group of objects. Partly, so you don't have to write code in the objects that create sub objects and partly to not have to make so many create-commands (queries) in objects.
Deleting an objectIf an object is noted as being possible to delete in an object description, call delete. Note that the code might change it's mind and then you'll get an error.
Update managementAfter a change that might propagate to other variables, the data in the object tree is refreshed. This is an expensive operation, so it might be turned off when doing a series of changes XXXX XXXX Turn on/off updates in UI-API.
Queries to users from object
DecoratorsThe UI-API doesn't need to concern itself with Decorators; they are invisible and information about them is not sent back. They are added to code by using the low level API or by defining them in Instances.
ProgrammingThis is how you implement the complexities of the model -- calculations and rules. There is no possibility of a user interface here. All the methods discussed here are documented in the top.pm pod, unless something else is specified.
Object IDsAll objects have Ids. They might be stored in scalars and are printable. Get an Object's Id with GetId() and retrieve an object from it's Id with GetObject(). Just store IDs as Links -- they might change! Some methods come in both shapes (like GetSupId(), GetSupObj()) and other are less orthogonal (e.g. GetSubIDs() and GetSubObjs, but they don't exactly do the same thing) so read the specification carefully. Any future version won't remove them, but might add new.
Overriding methodsLook at the Declarations chapter to see how to write methods and how to override the already finished ones. You'll override methods in top.pm, which is the top class for the hierarchy. (The Decorator top class dectop also inherits from top.) You can define your own methods but the only ones that will be called are those that has been defined by the API.
Using variablesTo get and set the values of variables, use GetVal() and SetVal(). Do not go to the object representation directly. (It might change in a later version, a subclass (/Decorator) might want to override the call and dependency tests might fail). There are also constants (GetConst()) which can only be specified in the declarations. Standard, ``normal'' Perl constants can be defined in the top specification file.
InstancesIntances are declared in the configuration files, see that specification. An Instance defines a small tree of objects that can be added as a group and can override default values for variables (and the alternatives for enums), define Links and Decorators for the objects in the tree. Instances aren't relevant for the programming API, except that objects will have to work after being created by Instances. Oh, OK, you could create your own Instances dynamically but it is probably better to keep your old similar, but less messy, hobby of doing your own brain surgery. :-)
sysObject, the other half of the APIThere is one sysObject for a created model. It keeps track of all objects and has lots of functionality (that shouldn't be in a kitchen sink like this :-( ). Some things are put put here to keep the top class conceptually less overflowing. To get the sysObject, use the method GetObjectKeeper(). There is a serialization API in sysObject.pm, see the pod documentation there, that saves everything that needs to exist between invocations. If you absolutely have to add data to the Object hash, and not store it in variables, you need to update SaveObjectParams() and RestoreObjectFromDescr() which builds the data structures that are serialized. This is not recommended; consider the ``Update refreshing'' mechanism...
Update after changesObjects in the model are often dependent upon other objects and their variables. After an object is added/deleted or a variable is changed it might affect other objects in some other place of the tree. These dependencies might go in chains. I.e. if a change in object A influences a sum variable in object B, something in object C. These dependencies need to be updated after changes. (The scratchvalue attribute for variables means it doesn't influence anything else; use it where relevant!) Finding dependencies like in a spread sheet would be hard to do automatically since this is , so Tree::Model use a brute force method. When a variable is changed or an object is added/deleted by the high level API, all objects in the model are refreshed (calling the SetValueAfterUpdate() method). All variables are monitored and if some variable is changed, the whole refresing procedure is done again(!). (This only applies to variables that haven't the attribute ``scratchvalue'' set to true.) This means two things... update is the probably the expensive operation -- and don't indiscriminately set a variable to a random number at every update -- you will get an error! You might want to turn off this expensive operation if you e.g. do block changes. Then you can do an update after all changes are made. Use XXXX.
Queries to users from objectThis is integrated with the UI-API, read that part first. When the model data is collected for sending to the UI-API, all objects are asked for any specific questions/commands that they want to send to the user. This is method GetCmdListForUI(). The object should send an array of CallBackQ objects. See the documentation for CallbackQ.pm for that. Answered questions will be sent back to the object by calls to XXXXX. The returned answers to the parameters will be stored XXXX. (See top.pm documentation.)
Links APIAgain -- there are no references between objects. Use Links instead, as documented in top.pm. All links are named by a unique string (if you want, use '0', '1', '2', etc). This means an object can have multiple links to another object for different functions. (See the name as IP ``ports''.) Add a link from an object to another with $o->LinkSet('Linkname', $objid_target); and get the Object ID of the target with my($id) = $o->LinkGet('link_name');. There are API calls (NumberOfLinksToMe() and LinksToMe()) to see which objects links to an object. If many objects links to one object, try to use different names for the links (links with the same name are stored in an array).
IntrospectionDuring runtime, information about classes are stored in a Tree::Model::SysDef object. To get the class definition for an object, call it's GetClassDefinition() method. This is not recommended. Most parameters in the class definition are accessible with methods like is_decorator(), GetAllowedBelow() and GetVariableDecl('variable_name'). These alternative methods returns the correct value if the default class value is overridden in the object from Instantiation or programming. To handle the variables in an object, there are two better methods in sysObject (See the documentation for allowed attributes, etc.) get_variable_attr($object, 'variable_name', 'attribute_name'> set_variable_attr($o, 'var', 'attr', $new_value>
Decorators
Overriding callsA Decorator ``hanging on'' an object can override the standard method calls to that object. (XXX Add list to documentation!!) The method handle_msg is called in the Decorator, with the parameters: $decorator->handle_msg($object_to_override, $method_name, parameters ... ); Return value is a two parameter list. The first is what to do and the second is a value (if used). If the first parameter is Tree::Model::NOOP (or undef), nothing will happen. Note that if the Decorator calls the object it will get called itself to filter the result of its own call! To call a method without it being filtered by Decorators, use $obj-_no_decos($msg, parameters ...)>. If a filtering Deco returns (Tree::Model::dectop::SET_RETURN_VALUE, $value), it will set the return value and the ``real'' object won't be called. This is how a Decorator could start if it wants to replace the return value for the GetVal() method when a variable named salestax is asked for: CodeOverride name=handle_msg code=<<ENDCODE sub { my($me) = shift; # Decorator object (me!) my($o) = shift; # The object this decorator sits on. my($msg)= shift; # What method was called. # @_ is now the parameters to the decorated object. return (undef, undef) if $msg ne "GetVal" || $_[0] ne "salestax"; # ... etc... if ( .. test .. ) { # Filter the result of the method: my($methodresult) = $o->_no_decos($msg, @_); return (Tree::Model::dectop::SET_RETURN_VALUE, $me->some_modifying_fun($methodresult)); } # This will replace the return value for the method with 0.4711 # and the method won't be called at all. return (Tree::Model::dectop::SET_RETURN_VALUE, 0.4711); } ENDCODE
Multiple Decorators on an objectMultiple Decorators can hang on an object. They make up a chain of Decorators that are called in turn, longest hanging first. If a Deco do return (Tree::Model::dectop::RETURN, $value), the remaining Decos won't get a chance to filter that call. If SET_RETURN_VALUE is used instead, any later Deco might override the behaviour. There is also support for a series of filtering Decorators. With a series of Decorators and if none returns Tree::Model::dectop::RETURN or Tree::Model::dectop::SET_RETURN_VALUE, then all Decorators that returned Tree::Model::dectop::FILTER_RESULT will be called like this: $deco->filter_result($obj_to_filter, $msg, $result_of_call, parameters); Where $result_of_call is either the result of the call to the object's method or the result of the previous filter. This means that one Decorator will filter the output of the previous one, etc.
A Decorator on multiple objectA Decorator can get an array with the Object IDs of those it hangs on by calling the method hangs_on().
Creating a new Decorator and adding it to an objectXXXXX Document and/or add method to create and add deco.
Deletion, especially when serializingThe method $deco-Removed_from_obj($oid_removed_from)> is called whenever a Decorator is removed from that object. If you override, don't forget to call SUPER. By default, the implementation delete the Decorator when the last object it filter is removed. If you don't want that, just call
DeclarationsThis describes how to write a specification for a system in Tree::Model.
Formatting the data filesComments in the configuration files start with a '#'. Pod works too. The format for a command line is: CommandName param=value param2=value2 \ param3=value3 etc Some commands doesn't take parameters. Case is irrelevant for commands and parameter names but not for string values. Note the ``\'' at the end of the line to make a continuation. If a parameter is a string with spaces, it must be quoted like "this is quoted". Bare words is otherwise ok. To define an integer or a numerical value, just write it out. Boolean values are true/false (or 1 and 0). Lists are written like this: TestCmd param=( 1 2 3 hej svejs "foo bar" # These are frotz gazonk ( 2 3 4) # legal comments. 42 4711 ) Note that the list has a list as an element. Parentheses are used around lists and new lines in a list is legal. (And, yes, it was a total waste of time to write a config file package when there are better alternatives out there. It was an exercise; one of the standard jobs I do in all languages and I hadn't done it in Perl yet.)
Top level specificationThe top level system specification file, named by default spec.gconf, declares the system name, what packages to use and e.g. what classname path the generated code should have and which have names of all the files containing declarations. There is (minimal) support for pod and comments are sh-like. If the data files have just a name and no path, they are assumed to be in the same directory as the top specification file. The files are read in the listed order, so files that just adds to classes (e.g. to use specific config files that adds code) should be last. A typical spec.gconf looks like this: # This defines the bla, bla, bla # Please note that simple pod is ignored: =head1 Foo foo bar =cut DefineSystem name=CW version=0.11 files names=( topdef.gconf adef.gconf codedef.gconf instdef.gconf util_foo.gconf) use name="strict" use name="warnings" usePackage name="POSIX" # Want POSIX::ceil(). use name=FOO constvalue=42 # Defines a constant EndSystemDef
Creating a model for loadingThe specification files are used to generate a .pm file which is use:d to run the modelling application. An example is found in the first test (.t) file. If you stand in an unpacked distribution, you can generate a loadable module with this shell command: perl -Ilib -MTree::Model::CompileSpec -e ' $sys = Tree::Model::CompileSpec->new("t/EX1", undef, -10); open(UT, "> foo.pm"); $sys->dump_spec(*UT, "A::B", "Tree::Model::top", "Tree::Model::dectop"); close UT;' Tree::Model must be installed before doing require "foo.pm";. To avoid name clashes, a prefix is added to the class names and a variant isa() method is supplied to make the naming transparent. The prefix is A::B in the example above. A class named Eg would be A::B::Eg. There is no hieararchical subnaming of classes (i.e. what class Eg subclasses doesn't change that it is blessed into A::B::Eg). This was a bit of simplification, but you should not write code that depend on this class name wrangling; it might change in the future. Hint: If you add a version number to the class prefix you can load two versions of the same system and they won't overwrite each other's packages.
ObjectsClasses are defined in specification files. Here is an example of two object specifications: ClassDef name=XLeg sup=PlasticMetal longname="Series X table leg" \ type=abstract # Declarations of variables that are inherited. DefVal name=color type=enum inherit=true \ enumvalues= ( white black mauve walnut dark_brown tacky_brown cerise steelblue pink green) \ lusermod=true default=black # (Etc) EndClassDef ClassDef name=BillyLeg sup=XLeg longname="Table leg, Billy" \ allowbelow= ( tablefeet _0-1 ) # Declarations of what groups objects of this class belongs to: groups names=( belowtable ofPlastic ofMetal ) # Etc. EndClassDef (The groups command is discussed with the allowbelow attribute to the ClassDef command.)
ClassDef attributesThe ClassDef command starts the declaration of a class. Most data about the class is defined there.
VariablesVariables are declared inside an Object declaration. Examples of possible variable declarations: DefVal name=cost type=int default=0 inherit=true \ ShowPriority=normal min=0 DefVal name=weight type=int inherit=true \ ShowPriority=( 0 low default normal) DefVal name=Comment type=str lusermod=true inherit=true \ userinfo=true description="Your own notes." \ scratchvalue=true ShowPriority=("" low default normal)
DefVal's attributesThis is the variable declaration pragma in a class declaration.
SetDefaultValueThis is a command inside an Object specification. It is used to override defaults of an inherited variable. setDefaultValue name=link_aimed value=true inherit=true inherit has false as default. If not changed, a subclass will get the original value for the variable. Variable attributes that might be changed are value, lusermod, enumvalues (if type is enum) and showpriority. In practice it has the same effect to reset the value as redeclaring the variable with the same name but with a new value.
DefSumvalueThis is a variant variable declaration. It is not inherited. If an Object has a sum variable, all objects below in the later build (not the inheritance!) hierarchy get traversed and all values in a field is summed and stored in this variable. ...
DefConstTo define a constant value: Const name=strength_cost_perc type=list value=(-20 0 50 100) Constant names live in a separate namespace (i.e. the normal GetVal() call won't get the value of a constant; you have to use GetConst(). You can't use constants with the same name as variables anyway. Constants can't be user visible but they have the attribute inherit. And, no, you can't override them.
Defining a methodSee the documentation for top.pm for what methods might be overridden. To add your own methods, just override a method named as that. (The format for specifying long parameters are usable for other string variables.) CodeOverride name=SetVal code=<<ENDCODE sub { my($o, $n, $val, $who) = @_; if ($n eq "cost" && $val == 0 && $o->do_check_of_Bar($n) ) { my($spikes) = $o->FindDirectSubsOfClass("ChairSpikes"); return "" if scalar(@$spikes) > 0; } return $o->SUPER::SetVal($n, $val, $who); } # Just a help routine. Not that they can be added like this! sub do_check_of_Bar { my(@o, $n) = @_; # etc return 42; } ENDCODE
Adding to ObjectAn Object declaration can be split into two. There is a command AddToClass name=.. which adds definitions to a class (that has already been seen; see order of files in top specification file.) That the Ojbect has been specified in multiple bits is transparent. This is used for having object's code in a separate file. This might make the declarations clearer. Example: AddToClass name=WheelGuard CodeOverride name=SetValueAfterUpdate code=<<ENDCODE sub { my($o) = @_; $o->SUPER::SetValueAfterUpdate(); # Any stuff superior classes need do $o->SetVal("armorcost", 0); # (See WheelHub explanation) # Check if constant 'maxarmorweight' exists and set error message to # user if we are overweight. my($max) = $o->GetVal("maxarmorweight"); my($weight)= $o->GetVal("weight"); $o->SetErrorState("Too heavy! Max $max lbs (now $weight)") if defined $max && $weight > $max; } ENDCODE EndClassDef
InstancesA Instance that can be instantiated to create a new model is declared something like this: DefInstance type=Kitchen name=RetroSteel topobj=true \ description="Stainless steel kitchen; retro old science fiction" \ allowbelow=( *Chairs *Tables Sinks _1-2 General KitchenUtils ) # Override values for variables: SetVal name=fullname value="Stainless Kitchen; 60's space" SetVal name=designer value="ACME Design team" SubObj type=MetalRefrig name=Refrig1 SetVal name=chromed value=true SetVal name=size value=xa_large # (Price is automatically set from the above values.) SubObj type=Paint name=Moonwalk SetVal name=technique value=spraypaint SetVal name=price value=2000 EndSubObj EndSubObj SubObj type=KitchenGarbSteel name="Garbage Storage" # This is a link. The code looks for if a link with that # name exists and then ... MakeLink name=GCrunch objname=Sink EndSubObj SubObj type=SS_Sink name=Sink SetVal name=garb_disp value=true Decorator type=SinkFilter name=remove_X # This error will be set for the sink in a SSteel kitchen # if the rest of the code sends XXXX. SetVal name=SS_SinkError value="Illegal operation on !" EndDecorator EndSubObj EndInstance As is obvious, EndInstance finish a declaration. The SetVal Command is used to set a new default value for a variable. It can override the allowed values for enums with enumvalues and define showpriority. SubObj defines an object hierarchically below other objects. It can have allowbelow defined. The type attribute sets the class name of the top object in the Instance hierarchy. See the documentation for doAllowSpec.pm for how to write allowbelow specifications. XXX Checka 'longname' for Instance! Is it a noop??
DecoratorsThe highest class in the Decorator hierarchy is declared like this: ClassDef name=DecoTopsy sup=dectop type=decorator A ``normal'' Decorator is declared like this: ClassDef name=DecoTest sup=DecoTopsy type=decorator The other changes regarding Decorators are mostly what methods they inherit and override. It is described in the Programming subchapter.
Back to
Meditations
|
|