|Perl: the Markov chain saw|
I enjoyed your node very much.
One observation - in the case of the sciences, we spend a great deal of time bringing people up to speed so that they can be able to manipulate the abstractions we use to describe the world. I don't see this happening in the IT world of *** certified technicians, developers and the other short cuts that are used to force feed basic knowledge and practices into someone who then calls themselves a certified programmer, admin or whatever will sell a certificate.
A PhD can take three to seven years and even then we regard the output of these programs as beginners - and this is someone who has been constantly exposed to and practising the discipline since graduating secondary school. It takes several more years to really know what is going on and to be able to manipulate it in a peer acceptable way. On top of that we promote our research and ideas by writing down and publishing the results of our research in a specially crafted format that allows some one to reproduce what we have done. If it's not published and reproducible, its simply not science. Nowadays if it's good science you count the numbers of times it is cited by others as support for their own work.
The closest I think IT comes is in the likes of APIs, RFCs and formal documentation systems. Although certification is very in vogue, really the products are still beginners - very like moving from printed letters on ruled pages to joining the letters up in scripts on unlined pages, but still really not knowing how to compose an essay very well. The IT equivalent of peer review is either passing and failing test suites or use cases. Not many organizations do code reviews from what I can see. And each of these does it differently - I have never really come across a standard for writing code. Or seen it enforced in the way the scientific community uses peer review to at least standardize the transmission of knowledge in papers.
The closest I have ever seen of the process of teaching and learning how to do the trade in IT was in my company before it was acquired by a larger corporation. They brought in people, had them work under more senior managers and they learned through tutoring and experience how to manipulate the codebase in a way that was acceptable to groups of coders. We had documentation of the code both in the source and as formal documentation. We had test suites, though not to the level I see advocated by the Monks. The process was sufficiently succesful enough that we could offshore to India, but the documentation was not enough - what was still missing was the simple ability to ask a question and get an answer. Once the original coders were gone, development slowed to a fraction of what it used to be.
Its not a problem of the new developers - they are very skilled. Its a problem of how much time it takes to develop enough knowledge to understand an abstraction in the code base that is not documented well or at all. From what I can see we may all speak Perl but there are dialects and accents in how it is used. This is much the same as in natural languges where there are dialects of English I can't begin to understand and accents that I find more pleasing than others.
Going back to you original point, I think what you are advocating is great if it's your own stuff, you are working within a stable group or you are very experienced in all idioms of yor language/development platform. The main problem is the level of documentation needed for projects where the personnel are not stable and the next developer is someone who is not a script kiddie but is not as fluent as the last guy to comprehend and be able to make the modifications necessary to complete a job. What does some fellow with a small business really need to have in order to keep on using the code you developed for his web site, his aplication or whatever. The closest he can undestand to coding and development is going to his local garage and getting work done on his car. The proof of the mechanic is how well the car runs afterwards. An upgrade (new seats, new stereo) either work or don't work - the question for the mechanic is how well can he make modifications based on his understanding of what appears to be necesary to modify the car. And this difference between what you meant and what he thinks he needs to do to make a modification is where the dragons lurk. Just because he is not the world's greatest mechanic shouldn't stop him from changing the beaks, replacing sparc plugs or doing a new paint job. The question is how to help him recognise the job will be more difficult that first or second glance shows and then help him understand enough to either do it or pass it on to someone more experienced. Sometimes the manufacturer's manuals just don't do it and you wish you could quiz the systems mechanic on what the hell was he thinking.
I hold that if you write something you should keep a mind to the next maintainer and provide enough documentation or test suites to help them understand what the code does. Sometimes it has to be code comments. Promoting standardized coding practices ala Conway is a good way of reducing dialectic confusion. But if you think the code needs it, then put in the comment. The new guy will soon see whether they are useful or not, but at least you gave him the chance to understand what you were thinking of at the time you wrote the comment. If nothing else it shoes some fairness to the boyo that paid you to do the work in the first place, even if he was a tight wad.