Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Re (tilly) 3: (OT) Rewriting, from scratch, a huge code base

by tilly (Archbishop)
on Sep 30, 2001 at 17:14 UTC ( #115724=note: print w/ replies, xml ) Need Help??


in reply to Re: Re (tilly) 1: (OT) Rewriting, from scratch, a huge code base
in thread (OT) Rewriting, from scratch, a huge code base

Colour me unconvinced.

First take the Netscape/Mozilla project. The article addressed that one and said that the decision to rewrite was an unmitigated disaster. And implies strongly that if they had decided to work with the existing code-base, they would have had better results. Well the way that I remember it, they were getting eaten alive by IE, and development was crippled by having to deal with and work around layers of bug fixes on bug fixes. The fact that one set of nightmares came true doesn't mean that the other would not have. Hindsight isn't 20/20. Rather it is speculation with the comfort of knowing you will never find out if you are wrong.

Take next Perl 6. Perl 6 had many major goals. The most important was to reinvigorate the Perl community. Others included making it easier to get into the internals and easier to port to different platforms (eg the JVM, C#, or a more aggressively optimized binary). Note that Perl 5 was not doing too well at these tasks despite much energy and interest. Well Perl 6 has done quite well at the first, and I am fairly confident that it will be able to succeed in the others. It just won't do it on an aggressive schedule.

But I said I don't like to argue from failure. How about a success? Take a look at perl. Scan for the words "version 5". Perl 5 is a complete rewrite of Perl 4. According to Joel that was a horrible mistake, and Perl 5 was bound to fail. Didn't fail that I see. In fact when I take a look at what it resulted in, I don't think that features like lexical scope, removing 2/3 of the reserved keywords, adding references, etc, etc, etc would have happened in the same timeframe doing incremental refactoring. Furthermore I give Larry Wall due credit, he has probably been writing influential free software projects longer than both of us have been writing software combined. Given his trail of successes, when he thinks a rewrite is doable, it probably is. If he thinks that it is a good idea to get where he wants Perl to get, well he is the one whose vision got it where it was.

Now this is not to say that refactoring is a bad idea. When it works, it works well. It is a useful tool. I am glad that it helped you on File::Find. But I think that most big projects can profitably use multiple modes. For instance the Linux project does both. Most of the time you do incremental ongoing development. But I think that ESR made the right choice when he made CML 2 a complete rewrite. Sometimes you incrementally adapt a component. Sometimes you replace it. You are doing something wrong if you need to replace big components very often.

And one final thing. Ovid is dealing with a system, one of whose problems being that it had a bunch of features added without much rhyme or reason because an ex-employee thought they were "cool". It does not have a large user base. I don't think, therefore, that he should build tests based on the current behaviour, enshrining the misfeatures in tests. Rather he should do some research about how the system is actually used, and only test for what people use from it. Whether or not he rewrites from scratch, blindly refactoring based on the current behaviour will not solve one of the problems that he wants to solve. And, whether or not he rewrites, he should think about how to solve the business problem. Perl 4 did not stop development just because Larry was working on Perl 5. Perl 5 is not stopping active development just because Perl 6 is being worked on. Creating great software is one thing. But you need to survive to actually do it...


Comment on Re (tilly) 3: (OT) Rewriting, from scratch, a huge code base
Re: Re (tilly) 3: (OT) Rewriting, from scratch, a huge code base
by chromatic (Archbishop) on Sep 30, 2001 at 21:56 UTC
    You're right, we're all speculating with the benefit of hindsight. Probably no one at Mozilla thought they'd spend three just to get something that could honestly replace Netscape 4. (This has suddenly become a Software Quality Rant.)

    I would like to see programmers give up the idea that the only way to understand code is to rewrite it. It's nice that new programmers have the energy and inclination to start things from scratch, but we all know where the elephants go to die .

    Maybe I'm going too far, but I doubt we'll see quality software until programmers learn how to improve *existing* software. Part of that means writing maintainable software, yes. Ovid's got it exactly right, above.

    But if they only know how to rewrite, not maintain, there's little hope of that.

      I agree that it is good for people to learn to use and maintain wheels instead of building them. I also agree that the experience of maintaining code teaches you a lot about how (not) to write maintainable code. However there are at least three good reasons I would cite for when it is important to rewrite code:
      1. You need something available in the language you are working in, and the original code is not. The example I cited above was rewriting code in Perl that had been in Expect scripts. Without the rewrite you cannot add consistent error handling nearly as easily.
      2. The component has been identified as exceptionally buggy. This is based on research at IBM where they found that they could get impressive improvements in code quality by tracking bug count, and then rewriting from scratch the worst 10% of components. The point being that the components usually had some fundamental design flaw, and your demonstrated odds of writing a better component were very good.
      3. You have completely reconceptualized how the component should work, believe that the benefits will be very large, and accept the costs of an attempt. This is the most dangerous reason, it is easy to fool yourself. However sometimes dramatic change in how something fundamentally works is easiest achieved through a rewrite. (Perl 5 as opposed to Perl 4.) Even so, I think this is an absolute no go unless you make it a priority to support the old model from within the new. The new model needs to not only be better than the old, it needs to subsume it.
      But note that while I think there are cases where rewrites are justified, I freely and absolutely agree that people tend to launch into rewrites without sufficient justification...

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://115724]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (7)
As of 2014-12-21 19:38 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (107 votes), past polls