|Perl: the Markov chain saw|
It's always interesting when I see a good programmer espousing something with which I strongly disagree. More interesting, really, than reading things I agree with; it makes me consider my own position. Sometimes I wind up changing my opinion, sometimes the other fellow (or lass) does, and sometimes we come to recognize a fundamental difference in premises, which usually means we've come to understand the whole problem better. I ++ you for starting an argument, in the best Socratic sense of the word. :-)
The practice you describe I have called 'storybook programming', and I have gone so far as to tell people to avoid it. Each subroutine is only called once, and the main part of the program consists in calling each of them in order. It's very much like a storybook with a table of contents and chapters:
The Accountant and the Ledger
Chapter 1: Wherein our Hero gets the Expenses
Once upon a time there was a little accountant named Irwin, who lived in a cubicle. One day the Lord High Fiduciary visited the cubicle, and asked Irwin to create a financial report. So Irwin opened up the Big Boring Book of Expenses, and...
You get the idea. The chapters describe what happens, in the order in which it happens, and are mostly there to divide things up by topic. There's never a point in a storybook where it says, 'And Then he wrote another report... please go read Chapter 3 again.' And the chapter titles seem to describe what goes on in the chapters... which itself is a point of dread for me. Like comments, reading subroutine names is no substitute for reading code, particularly since things, as you've noted, tend to change over time.
I find that novice programmers often think this way, and that, for the naive, it leads to a predictable set of problems which are not wholly unrelated to the point I'm trying to make to an experienced programmer.
Because the subroutines describe the events in sequence, rather than the events which happen repeatedly, they'll often have redundancy... Irwin does the same thing in chapter 10 that he did in chapter 2, for instance. It's like unrolling a loop without a good reason; the result is awkward and unnatural.
Another consequence of 'storybook programming' is that there is a strong temptation to use global variables. The novice understandst that Irwin needs to deal with @expenses in most of the chapters, so he simply deals with it directly.
Soon someone tells him that he should be using strict, and so he does, and everything breaks, and he eventually solves the problem by turning the globals into package-scoped lexicals, which he declares at the beginning of the program, though he doesn't quite see why that was helpful.
Then he's told that he should really be passing these variables along as parameters... and so he begins the tortuous task of passing the same 20 variables along to every subroutine in the program.. and if he's got any sort of intuition, he realizes his program is getting steadily worse, for having taken all this good advice.
Now the problem, of course, is that the programmer is looking at the problem in a less-than-helpful fashion. Sure, he's divided up the task into parts... but those parts don't reflect the aspects of the problem which are important to him as a programmer. It's as if a surgeon-in-training were analyzing a human body by saying, "Well, there's a top half and a bottom half, certainly. And there's a left half and a right half... the body must have both of those, or it's not complete." These things are true, of course, but it's simply more useful to divide the body into functional units, such as organs.
Likewise, a programmer would do well to recognize that when a single task, or closely related set of tasks, is done multiple times, it is a good candidate for being enshrined as a subroutine. It is my opinion that, most of the time, things which are done only once are better off not separated into subroutines... though I'm certainly not dogmatic on the point. I've seen cases where it seemed aesthetically sensible.
But you're certainly not a novice programmer. One would expect you to have a pretty good idea as to how to structure a program... indeed, structuring programs is the point of your post. So what's the point of mine?
Well, what you're doing strikes me as being something like premature optimization... guessing as what's going to be needed down the road. As you note, project requirements change, and features are added; and the thing that makes that a challenge is that you don't know what's coming. In making a 'naturally' linear program non-linear, you've made a guess at which tasks might need to be repeated.
If things change, and you guessed correctly, all is well. On the other hand, what if the way you divided up the problem turns out not to reflect the nature of the new problem? For instance, say you have to accomplish tasks A, B, C and D once each. So you write a program that looks like this:
Now, if the project changes, and you have to do A and B more than once, or C and D more than once, you're set. But perhaps task E gets added, and you need to do A to get ready for it. So you say:
Now it seems A should have been a subroutine. And then it becomes necessary to perform D multiple times for each C... suddenly you're refactoring a problem that didn't need 'factored' in the first place. At this point, life would be easier if you had some linear code you could simply sequester into the subroutines which are now obviously appropriate.
Now, this description is rather abstract, and I'll have to beg your pardon for not supplying concrete examples... at the moment, I'm pressed to finish this post and do other things. In light of real examples, I think several things would likely become clear. One of which is, there's a measure of common sense involved. I'm not suggesting that a programmer should strictly refrain from making reasonable guesses as to what has a high probability of being needed tomorrow. But I do think there's a danger in it and I think there are good reasons to prefer your original, simple program. In my opinion, the solution should fit the problem... and linear problems deserve linear solutions.