Re^5: Some thoughts around the "is Perl code maintainable" discussion

[paddy3118]

Yes, I do know that other languages have obfuscation contests.
I would stand by what I said about what the community views as cool bleeds into the communities practices at work.

With Perl, I always get the feeling that maintainability is seen as a bolt-on afterthought that has only a very small impact on the languages design (its feature set and syntax). Don't be surprised if the average user does the same.
- Paddy.

--
    Debugging is twice as hard as writing the code in the first place. 
    Therefore, if you write the code as cleverly as possible, 
    you are, by definition, not smart enough to debug it.

    —Brian W. Kernighan, co-author of The C Programming Language and the "K" in "AWK"

[BrowserUk]

I would stand by what I said about what the community views as cool bleeds into the communities practices at work.

It is (or was, I don't keep track of such things), 'cool', to wear trainers without laces, but I never saw a marathon runner, nor sprinter, nor footballer, nor even a local jogger adopt this form of cool. 'Cool' is doing the opposite of what our parents did or want us to do. Or the same as the celebrity we admire. Or doing different to that group of people we wish to distance ourselves from.

Cool is subjective, transient, divisive. Cool is undefinable beyond a small group of like minded people. Using it in a discussion to 'tar' a whole community with a single, personal perspective is emotive, irrational, oneupmanship. Ie. Seriously uncool.

With Perl, I always get the feeling that maintainability is seen as a bolt-on afterthought.

Maintainability cannot be "bolted on", just as it cannot be designed in. It has little or nothing to do with statement-level or block-level syntax or semantics.

It has everything to do with good algorithms and their clean and concise implementation; clear abstractions and clean interfaces; complexity management and the reduction of interdependencies. Good design at the application level, and formalised documentation and working practices at the project level, are always far more significant in the costs of ongoing maintainability than statement level syntactic choices of a given language, or of the language choice itself. Always.

... [Yet another smart quote from a smart person, quoted out of context to support an undefined opinion]

Quoting smart people does not make you smart.

And here we get to the crux. When you say so little and imply so much, you are most often concealing your own preferences, rather than conveying a supportable argument. The implication from all the things you are not saying, is that you consider Perl's permission of idiomatic construction to be clever and therefore unmaintainable.

That is akin to arguing that the componentisation of manufacturing, eg. large scale integration of transistors into ICs in the electronic industry, is unmaintainable when compared to say discrete component constructions from 30 years ago. And to some extent you are correct. A single failing transistor in an IC is impossible to replace, so maintainability is lessened. However, ICs are far, far more reliable than discrete circuits, so the need for maintenance is far, far less. Overall, we have a win.

A similar analogy can be drawn between 1970s, normally aspirated, mechanically-timed and ignited car engines when compared to todays fuel-injected, variably-timed, multi-valved, electronically ignited engines. In the 70's and early 80s, I (as an amateur) was perfectly happy to undertake rebuilding a car engine at home with simple tools. These days, I wouldn't know where to start. However, modern engines are far more reliable, require far less maintenance, and have much longer service periods than those old ones. Again, an overall win.

Perl's idiomatic constructs allow the concise representation of high level algorithms in a few short statements. Just as one statement of C represents many lines of assembler, so one line of perl can take the place of many, many lines of C. To argue that the use of idiomatic Perl is 'too clever' and therefore unmaintainable, is to argue that assembler is more maintainable than C.

I've seen people argue that

for my $thing ( @groupOfThings ) {
    if( $thing <= SOME_CONSTANT ) {
        $thing = $thing * SOME_CONSTANT;
    }
}

Is "more maintainable" than

$_ <= SOME_CONSTANT and $_ *= SOME_CONSTANT for @groupOfThings;

because, they argue, at some point in the future, it might be necessary to do something extra within the body of the loop, and then the maintenance programmer would have to reconstruct the loop to the second form. That would be a 'bigger change', therefore 'more' or 'harder' maintenance.

I say rubbish. If it becomes necessary to re-factor the latter concise form into the former, then the algorithm has changed significantly. And the issues of maintenance--what other effects that change of algorithm will require and what knock on effects that has on the wider code-base--are far more important that the small effort required in performing the refactoring.

One of the things that is conveyed by the concise form is: this is a single cohesive step in the algorithm. And that of itself is valuable information to the maintenance programmer. Spreading that single, cohesive step over several lines in order to accommodate future changes that may never happen, discards that information and potentially raises doubts in the maintenance programmers mind.

My favorite cookbook often starts the SAUCE section of recipes with:

• Make a basic rue using 50 grams of butter.

  If you do not know what a rue is, or how to make one, you're stuffed right there. Except, if you look up the index of the book, it shows you how to make a basic rue, along with 20 variations. There is no need to reiterate this information in every recipe that uses it, because you can go look it up. After a few times, you don't need to, so now all the extra space that would be taken up by expanding that information in place everywhere it is used, would be redundant.

  This is an exact analogy of perlish idioms. The first few times you encounter them, you have to look them up. And then, you don't have to any more.

[blazar]

These days, I wouldn't know where to start. However, modern engines are far more reliable, require far less maintenance, and have much longer service periods than those old ones. Again, an overall win.

I'm not really sure: while certainly modern engines and other mechanical stuff have several undoubtful advantages over old ones, I think that these tend to vanish when you take maintenance and duration into account.

It's not about the engine... but a pair of years ago while driving my dad's relatively new Ford Focus I happened to have one of my passengers open the glass beside his seat: as soon as he touched the button, the glass fell down like a guillotine. It made such a noise that I reacted angrily, thinking that he had opened the door and hit something: in fact it was winter and very cold, I had just raised the heating to the maximum for a few seconds because the front glass was becoming completely foggy. Well, apart the disappointment for the fragility of the mechanism, I thought in that situation that it would have been fine if there were an emergency one in the form say of a leverage system behind a removable panel in the door just to temporarily cope with such an inconvenience.

Other (counter-)example, not having to do with cars but with refrigerators: here, in our "country house" (it's not, really, but I couldn't come up with a better expression) we have one that has undergone the moving through two houses and is some 35 years old! I bet it's not terribly enviromentally safe, nor efficient nor nothing. But it does work! A newer one we got only few years ago broke down the last month, and to repair it would have costed more than buying a new one.

I think that what's expressed by the former two languages applies to most material devices: they're seem not to be made with duration in mind. Whether this is a circumstance, and a side effect of some other win they may have, or a precise design strategy is debatable. I don't think this could be translated to programming languages, though.

because, they argue, at some point in the future, it might be necessary to do something extra within the body of the loop, and then the maintenance programmer would have to reconstruct the loop to the second form. That would be a 'bigger change', therefore 'more' or 'harder' maintenance.

I say rubbish. If it becomes necessary to re-factor the latter concise form into the former, then the algorithm has changed significantly. And the issues of maintenance--what other effects that change of algorithm will require and what knock on effects that has on the wider code-base--are far more important that the small effort required in performing the refactoring.

Well, yes: it's called YAGNI in brief, but you surely knew.

[paddy3118]

If 'cool' is too subjective, transient, and divisive; then feel free to substitute 'fun' in my original sentence, but you might loose that sense of novices looking on at what the masters do and wanting to emulate them.

Maintainance is enhanced by composability and the reduction of special cases - things that can be affected by language syntax.

I'm glad you brought up the IC design example, it enhances my argument. There are many ways to design your own transistor, AND gate, or flip-flop, but the digital designer is NOT given the choice. there are very few types of basic gates at his disposal and through the base uniformity, tested methods of (de/)composition, testing, and training in manufacturability, the designer is freed to make such advances.
On the quote by Brian W. Kernighan, It is about debugging which can be a part of maintenance and writing clever code. I mentioned some very clever coding practices above it. I still think its a fair quote. I am aware of the pitfalls of a poorly used quote back-firing but would rather modify what you said to "Mis-quoting smart people does not make one smart", which causes me to quote rarely :-)

I do not argue that assembler is more maintainable than C, and know that you never said that, it was part of one of your examples. I could however take the first sentence of your paragraph introducing the example to highlight some of our differences of opinion:

Perl's many idiomatic constructs allow the concise representation of a high level algorithm in an unnecessary number of sets of a few short statements.
Furthermore, this is seen as a good thing in the community. I think this goes against maintenance, as does "Do What I Mean" rather than "Do What I Say".


Here are two concrete Perl features that make it harder to maintain Perl programs, not un-maintainable, but definitely harder to maintain:

 * The explicit use of references.
 * The method of passing of arguments into subroutines.

You can't mandate against their use in a style-guide and they can be used so deceptively that they get through code reviews.

- Paddy.

[BrowserUk]

Update: I couldn't find this last night, but as far as

the concise representation of a high level algorithm in an unnecessary number of sets of a few short statements.

goes, this beats any Perl example I've seen into a cocked hat.

---

By "method of passing of arguments into subroutines.", I assume you mean 'by reference', making that essentially the same 'problem' as "The explicit use of references". Reading between the lines of your mentioning those, and "composability", I assume that you favour pure FP languages.

So when you rail against Perl for these features, you are essentially railing against side-effectful languages. Ie. Every imperative language in existance along with a good number of FP languages, that allow call-by-reference and mutable variables. And yet, the vast majority of software in existance, and most new software being written, is being written in languages that allow both. This is an old argument and one that scopes well beyond Perl.

As for DWIM -v- DWIS. Here is an example of Perl DWIMming:

sub perms {
    return !@_ ? [] : map{
        my $next = $_;
        map{
            [ $_[ $next ], @$_ ]
        } perms( @_[ 0 .. $_-1, $_+1 .. $#_ ] );
    } 0 .. $#_;
}

print @$_ for perms qw[ a b c ];;
a b c
a c b
b a c
b c a
c a b
c b a

Take a careful look at that array slice: @_[ 0 .. $_-1, $_+1 .. $#_ ], and note that $_ takes the values 0 .. $#_. That means that in the first iteration of the slice, the indexes are 0 .. -1, +1 .. $#_. And at the other extreme it will be 0 .. $#_ -1, @_ .. $#_.

When I first encountered Perl and started using the range operator (..), I was disappointed that it didn't work for descending ranges. Do you see how Perl is DWIMming there, and how, if it always DWIS, then I would have to explcitly code conditional checks to ensure that the slice didn't address out-of-bounds indices. Whilst not hugely difficult to do, adding those checks and tests to the routine above would destroy its simplicity and clarity, and obscure the algorithm. And this is just one example of DWIM, that greatly simplifies the concise and expressive coding of algorithms in Perl.

Try a brute force conversion of the above to C and you'll understand how much complexity this well chosen 'irregularity' in Perl avoids. And Perl is full of these. Not that I agree with all the DWIMs, but then maybe I just haven't yet encountered the situations where they make sense.