Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask

How to avoid (or handle) module interface changes

by chester (Hermit)
on Oct 21, 2005 at 18:16 UTC ( #502089=perlmeditation: print w/replies, xml ) Need Help??

Hello monks,

Just some random thoughts about some things I've encountered recently due to some poor judgments on my part, and some thoughts about the concept of data-hiding.

At work I have a bunch of Perl scripts that access some central data. The data is a hideous mess and I'm unable to clean it at its source. Because I found myself re-parsing the same data over and over, I threw it into a module to attempt to make it easier to access; the module parses the data, organizes it and outputs it as needed. This is a good place to use OOP, I think: provide a cleaner interface to dirty data.

My problem is that the module's interface (the way it stores data internally, the way it's initialized, the way it sends data back to the user) is functional, but far from optimal. What I THOUGHT I'd need to do with the data is different than what I actually need to, for one thing.

Lesson learned: YAGNI! I first heard that term in the chatterbox a few days ago; if only I'd heard it 3 months sooner. Don't make guesses at what you'll need someday, because you're probably going to be wrong. Instead design something with separate, clean pieces that you can later put together to create anything you need.

Now, I know enough about OOP that I'm not directly accessing the module's guts in my scripts; I have accessor methods. But my accessor methods aren't the best. For example instead of returning a hash of hashes of hashes of hashes and letting the user deal with it, what I really should have been doing is returning an array of specifically what the user asked for. Doing that way would be good because 1) it's less confusing for the user, 2) the module can initialize/fetch parts of the data at a time as needed, instead of all at once, 3) it forces me to store the data in a cleaner way in the module, 4) it lets me change the backend without breaking the frontend. etc. etc. I wasn't doing quite enough parsing and data-simplification at the module level; using my module was easier than using the original raw data, but it wasn't as easy as it could have been. The bad thing is that as a result, my scripts are tied too tightly to the original modules' interface (or the structure of the data the module outputs).

Lesson learned: don't puke any more data at the people who are using your modules than they specifically ask for and require.

So the best solution is to AVOID interface changes. But barring that, assuming a change is required or greatly desired, the quesiton is : How do you handle interface changes when a lot of people are still using your old interface?

There are various ways I see of correcting this problem:

  • Require all your users to rewrite their scripts using the new interface. Theoretically the best and most correct option, might be best in some cases, but not likely unless time has no value.
  • Re-use the old module as much as possible; wrap the OLD interface with the NEW interface, so both interfaces are still available. I see this sometimes on CPAN, with module Something, and Something::Simple beign a simpler interface wrapped around the old one. I really don't like this in my particular case, because my original base code is not good. Best not to build a foundation on shaky ground, in other words. It also eventually breaks down as a general rule; wrappers on top of wrappers on top wrappers introduces much more room for error.
  • Re-write the whole module, then wrap the NEW interface with the OLD interface, so that both are still available. I see this sometimes on CPAN too: "blah() included for backwards-compatibility". So you mark something as deprecated, keep the duplicate methods for a while, and then remove the deprecated version after some period of time.
  • Just make a whole new package, ModuleNew, which does ths same as Module but with a different interface. Have the two be entirely separate entities. Code duplication isn't fun though. See also the 973 date/time modules on CPAN.
  • Use module versions to differentiate. Pre-some-version uses one interface, post-some-version uses another. Let the user request the version they want. In my case, this would probably require two version of the same module to be installed; I don't know if this is even possible. And of course bugfixes in newer versions either have to be backported, or else people using the old version miss out. I've had very little experience dealing (or caring) about which version of a module is installed, other than keeping up to date with the latest version.

I'm unsure which option is the best. The answer is probably "it depends". (Isn't that the answer to all questions? (Obvious answer: It depends on the question.)) But do you have any other thoughts or experiences?

  • Comment on How to avoid (or handle) module interface changes

Replies are listed 'Best First'.
Re: How to avoid (or handle) module interface changes
by eyepopslikeamosquito (Chancellor) on Oct 21, 2005 at 22:00 UTC

    Some general advice on interface design:

    • Start by imagining the best possible interface, unconstrained by the practicalities of implementation; though you may need to temper this ideal based on the practicalities of implementation, start by imagining your perfect interface.
    • As pointed out by TheDamian in PBP, you must "play test" your interface for a while to find a good, usable, practical one. Writing tests first helps here. "Play test" it from several different perspectives: newbie user perspective, expert user perspective, and so on.
    • Deriving an interface from real use cases and YAGNI helps keep the interface small and simple (and avoid over-engineering).
    • Be patient. You are most unlikely to design the perfect interface the first time. After you have used the interface in anger for a few weeks, re-think it again.

    See also Ingy's "Swiss Army Light Sabre" - or, "how do you design your APIs?".

Re: How to avoid (or handle) module interface changes
by dragonchild (Archbishop) on Oct 21, 2005 at 23:26 UTC
    Since the new version is mostly likely to have a superset of the old version's functionality and better decomposed to boot, you will usually want to provide a compat layer for the new one that exposes the old one's API. A few caveats:
    • That API has to be perfect, including any bugs.
    • You have to be willing to force people to switch if they're asking for a new feature that's already in the new version. Don't backport to the compat layer.
    As always, be completely upfront. One important thing is to release a documentation update to the old version saying that this is obsolete and new development should target the new version. Then, point them to the compat layer and walk away.

    My criteria for good software:
    1. Does it work?
    2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?
Re: How to avoid (or handle) module interface changes
by rir (Vicar) on Oct 21, 2005 at 21:29 UTC
    You left out the option of your new code living in the same namespace: to extend and wrap and deprecate stuff in one module.

    Then you have all your code proximate and can keep your users and push them toward the new code. In your case it seems the pushiness may be tolerable since users will be attracted to the better interface functions. It also leaves you in a good position to refactor and reuse.

    Coming up with new names for functionality can be a pain.

    Wrappers are a simple way to extend and abstact existing code. I think your point regarding wrappers is valid but I feel that you are viewing them too negatively in this situation.

    Be well,

Re: How to avoid (or handle) module interface changes
by 5mi11er (Deacon) on Oct 21, 2005 at 19:22 UTC
    From your description of your particular problem, it sounds to me like a creating an entirely new module will be your best option.

    To eventually be able to replace the errant module, you might want to entertain a sub-name space that can be, at first, optionally "used/required", then later as the old interface is depricated, becomes the default. This is somewhat like the idea of wrappers, but possibly using inheritance to 'fix' the broken interface with a new version, but leaving the old one accessible as is for now...


Re: How to avoid (or handle) module interface changes
by swampyankee (Parson) on Oct 22, 2005 at 18:04 UTC

    I'm one of those rather peculiar programmers who would finds excuses to write library functions 8-), so I can understand why you write a module.

    I tend to like your 3d option (complete re-write, keep the old interface for the existing code, and make the new interface available for new code) the most attactive from a technical point of view, and your second option most from an overall point of view. I really do not like either of the last two options, which (in my opinion) are really different ways of doing the same thing.

    Added "attractive" so the sentence actually makes sense.


Re: How to avoid (or handle) module interface changes
by sfink (Deacon) on Oct 26, 2005 at 18:21 UTC
    Require all your users to rewrite their scripts using the new interface. Theoretically the best and most correct option, might be best in some cases, but not likely unless time has no value.
    I don't think you're giving this option enough credit. The issue is not whether time has value; it's whether you'll spend more time providing wrappers and backwards-compatibility shims than rewriting all code that uses the interface.

    I understand that it may be your users' time versus your time, but surely those aren't complete noninterchangeable? If you volunteer to go through all current users' code and update them to the new interface, then will they really have that big of an issue with it changing? And in the process, you'll learn for certain whether the new API really handles actual users' needs.

    Admittedly, this only works with a very limited user base (and almost certainly only in a closed-source situation), but I'm guessing that's the situation you're in anyway.

    In short: never pass up an opportunity to avoid the backwards-compatibility abyss. If you're going to write wrappers or adapters or whatever, be very very sure that you really need to and aren't just bending over backwards for artificial reasons. You'll be stuck with it soon enough; postpone the inevitable as long as you possibly can.

    Also, take a look at ingy's only for requesting specific versions of a module.

    I work for Reactrix Systems, and am willing to admit it.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://502089]
Approved by planetscape
Front-paged by Old_Gray_Bear
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (6)
As of 2018-06-19 12:54 GMT
Find Nodes?
    Voting Booth?
    Should cpanminus be part of the standard Perl release?

    Results (114 votes). Check out past polls.