Parse/Validate Code

[pos]

Fellow Monks:

I am seeking a way to parse modules in an effort to drive away the demons of runtime errors.

In the OOP method, one may end up with many packages, some of which get used in certain programs, while others are not needed. A single change in an object's method API will require many changes in other packages.

Currently, this can only be remedied by grep'ing the code to seek out and exorcise the offending ancient code. I seek a way to "compile" a given piece of code such that any discrepencies between a module's API and the code that is calling that module will be revealed at compile time. I call this compiling only because it should be smart enough to understand the perl language and match up a call to a perl module with the correct method. Of course we cannot expect to validate that the function is passing the correct order/number of parameters... however the ability to flag instances where the code will break if run is sufficient.

Perhaps someone has an even better way? Perhaps a module on CPAN has this capability already? I seek your advice.

Thank you
-pos

[davorg]

So if I understand you right you want to parse a module's file to work out what the module's API is and then parse any program which which use that module and flag places where the program uses the module in a way that's different to the API. Is that right?

If so, then you've given yourself a very hard (maybe even impossible) task. It's tricky enough to even _know_ what the API to a module is by looking at the file, let alone trying to write a program for it. Problems that immediately spring to mind include:

• Methods that get created on the fly using AUTOLOAD.
• Methods that are pulled in using Autoloader.
• Methods that are inherited from superclasses via @INC.

Even having identified the names of all of the object methods how are you going to identify the allowed sets of parameters? Remember that prototypes don't work for object methods. And what about methods that interpret @_ as a hash to simulate named parameters? Or parameters that are a reference to a hash or another object?

All in all I think this is probably impossible and you'd be better off finding a way to minimise your API changes.

--
<http://www.dave.org.uk>

European Perl Conference - Sept 22/24 2000, ICA, London
<http://www.yapc.org/Europe/>

[jeorgen]

One way you could do this is to never use other modules directly, but to have intermediate modules in between.

Something like this:

YourCode.pl--YourInterfaceModule.pm--ChangingModule.pm

So you create a stable interface towards your code, and when the interface breaks to the foreign modules, you've got one place to change the code, in YourInterfaceModule.pm.

Maybe there could be some kind of naming convention, so that when you want e.g. the functionality of Mail::POP3Client.pm (which according to the docs will change in that it will enforce named parameters in the future), you actually use your own module "MyInterfaces::Mail::POP3Client.pm", and then that module soaks up any interface changes.

update: I just realised that somebody with the right knowledge could create a generic module with:

• A slot for the object you really want to reach
• Some AUTOLOAD magic to forward any method calls to the object in that slot
• Then just write subs (methods) to handle interface changes
This could work something like this:

package MyInterfaceTo::The::Real::Module
use InterfaceModule ("The::Real::Module");
# InterfaceModule will handle method calls that fall
# through, i.e. a bit like Class::MethodMaker

# That would be all coding needed, as long as there 
# are no interface changes. When something changes, just 
# drop in a sub:
sub get_some_data {
   # Changed from param list to named params
   my $self = shift;
   my %param;
   $param{'shoe_size'} = shift;
   $param{'shoe_color'} = shift;
   $param{'shoe_type'} = shift;
   $self->object_slot->get_some_data(%param);
   }
1;

update II: After looking at TheOtherGuy's reply it became clear that the problem the fancy generic-interface-module I suggested above tried to solve, can be solved with less effort by just subclassing the module in question, of course (looking at the code now I realise: "Oh, I just reinvented inheritance!":-). However there was a deeper train of thought behind it: The generic-interface-module approach can be used to switch implementation (what module to use). If this is done at run time it would change it from an adapter to a factory, if I got the patterns in the Gamma book right.

/jeorgen

[TheOtherGuy]

It does seem that a bit more effort up front in design might go a long way. A couple of things I would suggest that have proven helpful to me:

The rule: NEVER CHANGE THE INTERFACE

Just accept it and design accordingly. I have never seen a case where the the interface needs to be changed; particularily in a way that breaks existing code.

Couple of things to try:

1. Have functions that take hash references. We use this all the time for cases that we know we will want to extend things later, or we want to make use of multiple overloaded parameters. As long as you ensure the default is equal to the old behavior, you even get to reuse all your old test sets.

$Stock->GetPrice( \{DATE => 'YESTERDAY', TICK => 'MSFT'}) || $Stock->H
+andleError();

2. Use inheritence. Create a new module, inherit the stuff you need, overwrite the functions that need to change. The API of the old module is unaffected, *and* you have implemented the new functionality to meet your current needs. This also works if you have to deal with someone else's design, which is a plus.

3. Limit the functionality; have a single function/method do a single thing. The cases where API's really need to *change* (rather than say, be extended by adding new functions) often are associated with having functions that do too many different things. Consider having more methods/functions with fewer parameters, rather than fewer methods with more parameters. Adding new methods does not break existsing code/test cases; adding new parameters does.

[Anonymous Monk]

It seems to me that the effort that you would need to expend to get this Perl Lint++ thingy to go is exhorbitant. IMHO your time would be better spent actually designing your modules and their API's before they are written and used. This (oft maligned) approach is a good way to minimise API changes and thereby avoid the problem.

[mikfire]

I guess I am not so talented - no matter how well I design my code ( and I do try ) there is always a point where I say "Hmm. I didn't think of that" or "The specs have changed and the code now has to do this" or "Boy, I was really hitting the crack pipe hard when I wrote that", etc.

I have written a few toys that are not perfect, but they basically grub through one piece of source code and pull all the subs out. It then grubs through a whole lot of other code and finds calls to those routines. It attempts to determine context then attempts to print out the full call ( my function calls frequently span lines ) and a line number. As I said, isn't yet perfect but it works pretty well. I have posted it in the catacombs. It is not well polished - it was never intended to be used by another person.

mikfire

Update: I modified this post slightly to add the URL

[pos]

I guess I am not so talented - no matter how well I design my code ( and I do try ) there is always a point where I say "Hmm. I didn't think of that" or "The specs have changed and the code now has to do this" or "Boy, I was really hitting the crack pipe hard when I wrote that", etc.

Thank you for the link to the code. Rapid prototyping is a very good development model for our business and Perl is the most flexible language out there. Our code has reached a point of reusability that i hadn't seen in Perl before and now i am looking to grow our toolset that we use to control it.

The API is very well thought out and designed, however we still need to depreciate the use of a function from time to time (much like perl itself =)

-pos
The truth is more important than the facts.
-Frank Lloyd Wright