Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?

Re^2: Nobody Expects the Agile Imposition (Part VI): Architecture

by roboticus (Chancellor)
on Jan 26, 2011 at 12:03 UTC ( #884328=note: print w/replies, xml ) Need Help??

in reply to Re: Nobody Expects the Agile Imposition (Part VI): Architecture
in thread Nobody Expects the Agile Imposition (Part VI): Architecture


Rewriting vs. Refactoring

If you start from a blank source file, I'd consider that rewriting, even if you start transplanting subroutines from the original back into the new file. I feel that taking a working system and transforming it without breaking it is refactoring, while building a new thing (even when borrowing heavily) from the original is rewriting. Of course, you can do both in the same project, as you may refactor some bits and rewrite other bits.

The problem I generally see with rewriting is that there's quite a bit of knowledge that's encoded into the system that's not immediately obvious. Things like:
  • Order dependencies: Systems consuming the output expect a particular sequence of records or operations and the order isn't documented.
  • Workarounds for problems in other systems: Sometimes bugs become part of the standard interface, and the documentation isn't updated. So writing fresh from the spec causes you to rediscover those issues.
Refactoring addresses these (somewhat) by making a change at a time. By writing tests for all the changes you'll hopefully capture some of this hidden knowledge in your tests. They won't necessarily be documented any better, but when you put it into production and other systems break, you should be able to trace back to the original test and document accordingly.

Of course, refactoring has its own issues. If you can imagine a cleaner structure for your program and try to refactor towards it, you'll find that "you can't get there from here", or you have to first go to Timbuktu before you can get back home.

With my experience (quite a bit), I find that the less you know about the domain or the application, the more you should lean toward refactoring. Similarly, the closer you are to being a domain expert, the more sense rewriting can make. The problem is that it's often difficult to objectively judge just how much knowledge you have about the domain. My approach is normally to find some dividing lines in the system where I can break it apart with the fewest changes possible. Then I can choose to refactor some chunks and rewrite others.

Overdesign and customer needs

You've pretty much hit the nail on the head: If I'm paraphrasing your arguments correctly, the difference between overengineering or not is largely communication. If you're going off and doing anything beyond what you've discussed with the customer, you're overengineering things. If you think the system needs to do something specific, or that the architecture needs to go in a certain direction, you need to have a talk with the customer about anticipated future changes so you can shape things correctly. So if you discuss things with the customer and get buy-in, then you're doing your job correctly. If you have discussed things with the customer, and they're adamant about a particular direction, then you need to do what they want, or you're throwing their money away.

(Gasp - they have a business need for a framework architecture?)

I've heard it mentioned that "functionality is an asset, while code is a liability". Too often, programmers know they need some functionality, but build their own rather than buying it. (I, unfortunately, succumb to this temptation a bit too often, myself.) I've been trying to periodically take a break from design and/or coding so I can sit back and review the requirements so I can stop myself from going off into the rabbit holes.


When your only tool is a hammer, all problems look like your thumb.

  • Comment on Re^2: Nobody Expects the Agile Imposition (Part VI): Architecture

Replies are listed 'Best First'.
Re^3: Nobody Expects the Agile Imposition (Part VI): Architecture
by sundialsvc4 (Abbot) on Jan 27, 2011 at 14:54 UTC

    To be quite frank, I believe that the essential difference between “refactoring” and “rewriting” is that one term is politically expedient, while the other term is not.   In both cases, you are doing the exact same thing in terms of the code:   you’re replacing the existing code with something altogether new, which renders the code inoperable (un-compileable) for an extended period of time and which must, in the end, be re-validated to verify that the new code works the same as the old.   The term, “refactoring,” is currently sexy and implies improvement ... “making an already-good thing better” ... whereas “rewriting” (wrongly...) implies previous failure.

    I do admit to the reality that, sometimes, in order to get approval and funding to do what badly needs to be done, you are obliged to resort to “necessary euphemisms.”

    Like it or not, computer software is very fragile (and therefore, costly) stuff, simply because it is riddled with functional and data dependencies.   It is, so to speak, “a house of cards,” which can only stand up to a very limited amount of “remodeling.”   I simply think that this is ... the essential and unavoidable nature of the beast.   It obligates us to try to do the best that we can, knowing that there are serious limits to that.   I submit that there is no silver-bullet language or technique to be had.   (He would rightly be a gadzillionaire who discovered it.)

    With regard to the point of “overdesign and customer needs,” there is the consideration that (a) the customer does not always know just where his business will take him; and (b) in any case, he is not a professional software builder and does not profess to be.   Sometimes you do need to “go beyond what you discussed with the customer,” because in your professional judgment as a software engineer, those additional elements (for example...) create the foundations for future characteristics of the system that are reasonably foreseeable as well as engineering-practical.   But, you need to be sure that you get all points about what the customer requests, and of what you have in turn decided to do, and every single subsequent change to the foregoing, in writing and signed-off and filed away for all eternity.

    Part of the (successful) argument for “frameworks” is that the cost of developing and maintaining them can be cost-amortized (or simply “unpaid-effort amortized”) among many projects that employ them ... thus allowing all of those projects to enjoy the full benefits without incurring the full costs.   The use of frameworks imposes a certain specific “world view” upon the project, however ... namely, the world-view of that particular framework’s designers, quirks and oddities and all.   Choose your project’s spouse very carefully.   The project’s entire future direction is necessarily molded around that of the framework, and in a very rigid way, except to the extent that the project’s actual implementation might be, by deliberate choice, architecturally divided into (framework-based) “client” and (non framework-based) “server” portions.   The cost/benefit analysis of using frameworks usually prevails in spite of this consideration, because so much of the constituent code in so many projects isn’t unique at all.

      In both cases, you are doing the exact same thing in terms of the code: you’re replacing the existing code with something altogether new, which renders the code inoperable (un-compileable) for an extended period of time and which must, in the end, be re-validated to verify that the new code works the same as the old.

      In my prior message, I mentioned that refactoring is a transformation without breaking the system. BrowserUk mentions later in this thread that refactoring is an iterative process, and he's spot on. For example, if I inherit a chunk of code with a mess of global variables and procedural functions and I want to transform it into an object-oriented system, then I'll do it stepwise with the system working after each step.

      First, I may decide to group related variables together into their own namespaces. So I'll create a new module for the new namespace, and put the variables in it. Then I'll update all the references to the variable to the new namespace. Then compile and test it. Then I'll move the functions appropriate to that namespace into the new modules, compile and test. After each individual step, the system still runs.

      It's true that the system needs to be revalidated after you're done. But you should maintain the system in working order throughout the process. If revalidation fails, then it indicates a failure of your test suite/data to cover all cases. So be sure to update your tests, too! Since you're doing it one step at a time, it's pretty easy to cruise through many iterations in a day. No iteration is drastically different than the one before it, but if you know where you're going, then your initial state and end state will be dramatically different.

      I could go on, but better authors than me have covered it well:


      When your only tool is a hammer, all problems look like your thumb.

      rewrite simply means write again, it doesn't imply any improvement, where as refactoring implies improvements in readability/maintainability/estensibility/test suite

        Actually, I think re-write in the Joel sense means even more than that.

        It generally means starting from scratch to achieve the same overall functionality, and usually, the same external interfaces. But with the express intention of improving the internal structuring and architecture, with the laudable goals of improving readability/maintainability/extensibility. This is often approached top down.

        Joel's point is that approaching these latter goals this way often goes ary, because in the process, many mistakes made and long since corrected in the original design and implementation, are made again.

        The refactoring process tends to be a bottom up, more constrained process, that improves r/m/x-abilities of individual functions or classes by cleaning up their implementations whilst retaining the fixes and extensions previously applied that lead to the need to refactor in the first place.

        Often, this type of refactoring needs to be applied as a multi-pass, step-wise refinement process to achieve best effect, but the advantage is that--done properly--you avoid large scale changes in any one step that are likely to cause extensive breakage.

        The difference between the two approaches is--or at least, certainly should be--far more than politically expedient terminology.

        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://884328]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others having an uproarious good time at the Monastery: (8)
As of 2017-06-25 21:48 GMT
Find Nodes?
    Voting Booth?
    How many monitors do you use while coding?

    Results (571 votes). Check out past polls.