Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Re: Programming *is* much more than "just writing code".

by MadraghRua (Vicar)
on May 09, 2007 at 17:21 UTC ( [id://614456]=note: print w/replies, xml ) Need Help??


in reply to Programming *is* much more than "just writing code".

I enjoyed your node very much.

One observation - in the case of the sciences, we spend a great deal of time bringing people up to speed so that they can be able to manipulate the abstractions we use to describe the world. I don't see this happening in the IT world of *** certified technicians, developers and the other short cuts that are used to force feed basic knowledge and practices into someone who then calls themselves a certified programmer, admin or whatever will sell a certificate.

A PhD can take three to seven years and even then we regard the output of these programs as beginners - and this is someone who has been constantly exposed to and practising the discipline since graduating secondary school. It takes several more years to really know what is going on and to be able to manipulate it in a peer acceptable way. On top of that we promote our research and ideas by writing down and publishing the results of our research in a specially crafted format that allows some one to reproduce what we have done. If it's not published and reproducible, its simply not science. Nowadays if it's good science you count the numbers of times it is cited by others as support for their own work.

The closest I think IT comes is in the likes of APIs, RFCs and formal documentation systems. Although certification is very in vogue, really the products are still beginners - very like moving from printed letters on ruled pages to joining the letters up in scripts on unlined pages, but still really not knowing how to compose an essay very well. The IT equivalent of peer review is either passing and failing test suites or use cases. Not many organizations do code reviews from what I can see. And each of these does it differently - I have never really come across a standard for writing code. Or seen it enforced in the way the scientific community uses peer review to at least standardize the transmission of knowledge in papers.

The closest I have ever seen of the process of teaching and learning how to do the trade in IT was in my company before it was acquired by a larger corporation. They brought in people, had them work under more senior managers and they learned through tutoring and experience how to manipulate the codebase in a way that was acceptable to groups of coders. We had documentation of the code both in the source and as formal documentation. We had test suites, though not to the level I see advocated by the Monks. The process was sufficiently succesful enough that we could offshore to India, but the documentation was not enough - what was still missing was the simple ability to ask a question and get an answer. Once the original coders were gone, development slowed to a fraction of what it used to be.

Its not a problem of the new developers - they are very skilled. Its a problem of how much time it takes to develop enough knowledge to understand an abstraction in the code base that is not documented well or at all. From what I can see we may all speak Perl but there are dialects and accents in how it is used. This is much the same as in natural languges where there are dialects of English I can't begin to understand and accents that I find more pleasing than others.

Going back to you original point, I think what you are advocating is great if it's your own stuff, you are working within a stable group or you are very experienced in all idioms of yor language/development platform. The main problem is the level of documentation needed for projects where the personnel are not stable and the next developer is someone who is not a script kiddie but is not as fluent as the last guy to comprehend and be able to make the modifications necessary to complete a job. What does some fellow with a small business really need to have in order to keep on using the code you developed for his web site, his aplication or whatever. The closest he can undestand to coding and development is going to his local garage and getting work done on his car. The proof of the mechanic is how well the car runs afterwards. An upgrade (new seats, new stereo) either work or don't work - the question for the mechanic is how well can he make modifications based on his understanding of what appears to be necesary to modify the car. And this difference between what you meant and what he thinks he needs to do to make a modification is where the dragons lurk. Just because he is not the world's greatest mechanic shouldn't stop him from changing the beaks, replacing sparc plugs or doing a new paint job. The question is how to help him recognise the job will be more difficult that first or second glance shows and then help him understand enough to either do it or pass it on to someone more experienced. Sometimes the manufacturer's manuals just don't do it and you wish you could quiz the systems mechanic on what the hell was he thinking.

I hold that if you write something you should keep a mind to the next maintainer and provide enough documentation or test suites to help them understand what the code does. Sometimes it has to be code comments. Promoting standardized coding practices ala Conway is a good way of reducing dialectic confusion. But if you think the code needs it, then put in the comment. The new guy will soon see whether they are useful or not, but at least you gave him the chance to understand what you were thinking of at the time you wrote the comment. If nothing else it shoes some fairness to the boyo that paid you to do the work in the first place, even if he was a tight wad.

MadraghRua
yet another biologist hacking perl....

Replies are listed 'Best First'.
Re^2: Programming *is* more than "just writing code" ( but it's *not* a science) (was: Programming is more than writing code, but it's *not* a science)
by mr_mischief (Monsignor) on May 11, 2007 at 22:10 UTC
    There is a serious flaw in comparing programming to scientific research. Programming in the large is not science.

    To equate programming in general to fields of science is like equating architecture and construction of a building to physics. Programming uses CS as a basis the way architecture and construction use physics as a basis.

    There are a couple of simple rules at work here that often get overlooked in the programming realm: The size and complexity of the project determines how sophisticated and informed your methods need to be. Also, the more levels of knowledge being built upon your contributions, the more fundamental and universal your contributions need to be.

    Let's face it. A cook is a chemist of sorts, and so is a candle maker. Neither is going to rewrite textbooks about chemistry. A carpenter or bricklayer has some sort of physics knowledge, but they are not going to split the atom or need a particle accelerator. At one time, a cab driver or a cashier actually had to know some math, but they weren't in the business of proving the number of dimensions in the universe.

    We don't have any solid, universally accepted differences in title between a sysadmin who's really talented with shell scripts and someone who writes frameworks for generalizing facial recognition software. Just because both might be called "programmer" or "software developer" does not mean the two are equivalent. I think this is a large part of the confusion in the software industry. I also think it's part of what causes so much friction between the highly qualified, very method-oriented people doing lofty things and the people who write small applications for small clients in small markets.

    The truth is, you don't want everyone to be a scientist. You don't even want everyone to be an engineer. You won't wait for a Pasteur or a Salk to mix your cough syrup. You won't wait for a Hawking or an Einstein to build your house, or for a Tesla or Westinghouse to wire it. What you want, down in the trenches, is people who take the results of research and engineering and figure out how to apply those components and best practices to the project at hand. Researching fundamentals is important, but so is having homes, food, clothing, and a community site like Perlmonks to debate the point. If the research and experiments have already given us evidence that a particular way of doing things is a good way, why can't some people just use those good ways while others further the research?

    Just because some people don't use best practices when they should doesn't mean everyone should be in the realm of developing new fundamental knowledge. That's where the line between technology and science lies. Technology benefits from science, but people as a whole benefit from the fruits of technology more than directly from the fruits of science.

    Day-to-day programming is, like building construction, more a technological trade than a science or research topic. It's a method of combining established parts using established methods to further one of a set of fairly common goals. Sometimes a building contractor finds an innovative and truly better way to build part of a house. An architect is more likely than the contractor to find a new and better way to design the whole house. That same architect might develop a new and better way to fasten two materials together, but an engineer is more likely to do so. The engineer might develop an altogether better material from which to make the fasteners, but a researcher is more likely to do that. It's this way in any field, really. I'm not sure why people get the idea that software should be different.

    Another trend you'll notice if you look at more established technologies is that these mostly stratified layers become more established the longer a trade is around. We didn't used to have a designer, a spinner, a dyer, a weaver, a cutter, and a seam sewer for making garments. People used to design and build a car single-handedly, but now there are auto researchers, auto engineers, auto factory engineers, auto factory employees, and auto factory robots. In time, I think we'll see such lines drawn in the software industry, of course with some mobility among layers. I just hope the titles aren't still, 'chief software designer', 'system analyst 3', and such meaningless chaff.


    Christopher E. Stith
      Excellent post. :-)

      I always liked the comparison between software and building construction. I think it's one of the best "real-world" analogies we can use to explain what we do, but there are differences.

      I view myself as being somewhere between an engineer and an architect. I like to "think with my hands" so to speak: think up a rough overall design, implement it and then move code around, rewrite parts until it works well and makes sense.

      The reason that works well (or at all) in software development is because of a pretty fundamental difference between building construction and software development; there is hardly any difference in software between the design and the product. It's only a matter of abstraction. Any really comprehensive software design is practically code (and you can argue that the reverse is also true). That's also a reason I don't like/believe in "automatic code generation from designs" - if your design is that comprehensive you're still programming, only chances are you're not using the right tools for the job.

        I'd just like to add that this high degree of similarity between a sufficiently detailed design and a sufficiently factored implementation was not nearly so much the case those few decades ago when I started programming.

        Languages and programming techniques have been moving toward allowing you to write a program by writing more design-like code while design techniques have been encompassing more of the details that are required when you write the code. There has also been more development of thorough methodologies that encompass both design and coding and these produce even more similarity in design and code when used.

        When I've done design that really pushes my abilities, I've longed for a tool that would let me build the framework by typing up some quick class definitions, change panes to see a graphical representation of the resulting relationships, make some changes in that GUI pane, switch back to the first pane and see the classes reflecting the results of my mousing, and continue back and forth like that. I've used some tools that try to come close to that, but none of them did it well enough to be a big pay-off. I always ended up avoiding much of the "help", using them in a "less integrated" manner, and being a bit sad that they were still too awkward or ridgid or whatever such that they became more of a hinderance than a help when I tried to make full use of them.

        Note that I agree with your final statement. Even if I had the perfect tool above, I'd only be using that for the design part of the coding. There'd be a line beyond which I'd then be coding and I'd stop using the design tool. Looking back, I'm a bit surprised to realize that C++ appears to me to have done a pretty good (but not perfect) job at effectively separating the design from the code where *.h is nearly all "design" and needn't contain hardly any "code" while *.cpp contains very little "design" (mostly just restatements of chunks of the design that is in the related *.h, and somewhat annoyingly verbose restatements at that).

        I do wish C++ allowed the "private:" bits to not be specified in the *.h, however. I also found that enforcing some rules about the order in which you write the different parts of your C++ class definition made it much easier to see the "design" when looking at the *.h. And this all makes me realize that I've not noticed this design/implementation break nearly so much in other languages. And I'll end my musing there (bringing the topic somewhat back to Perl and even to Perl 6, if one is paying proper attention). :)

        - tye        

        Thanks for the support.

        Like all symbolic language, there are inconsistencies in this metaphor. However, I find that sometimes a look outside the immediate surroundings of the topic is necessary. There's a psychological trick here in that removing one's thoughts from the topic at hand for a moment can help one focus more clearly once the mind returns to that track (train metaphor -- watch out). The fact that some semblance can be drawn helps serve the point, but it's far from the only reason for the comparison. I could have just as readily talked about how cute my step-nephew is for a paragraph before moving on to the topic, but I thought the metaphor could serve dual purposes.

        It's true that at some point comprehensive software design is almost code and that abstract code is nearly a design. I think there's another point that separates building construction and software development even more, though.

        I find that the biggest difference from building construction is because there's so little cost to moving code around. In building construction, it would be wonderful to pull a wall out here and put it back in over there. Unfortunately, there's a lot of physical labor involved, the materials are usually damaged during the remodeling, and the structure may not be as sound afterward. With software, it's much quicker and easier to move whole chunks of code from one place to another, change the interfaces, and generally muck about.

        This is in large part because software is abstract, but not necessarily because it is close to the same level of abstraction as the plan. Actually, I'd bet lots of architects are using the 3D building design software on the market to do with virtual copies of buildings what we do with code. Think of it as a visual integrated development environment using pluggable modules to create a value-added design by pointing and clicking, with certain details left to hand-written code(s)...

        Now that I've almost choked reading my last buzzword-riddled sentence, let me say that we'll probably never find a metaphor that fits programming better than devising a meal recipe. There are many ways to compare parts of software development to parts of other ventures, but it is so fundamentally different from everything that came before that nothing is particularly adequate in every way. Technical writing can be almost programming, and hardware design can be exactly programming, so those don't serve well as metaphors. A metaphor has to be enough alike while also enough different, and that's a fine line in any case.

        Strict research science is still not a preferred metaphor for software development (as a whole) in my mind. Part of the field is certainly science (or close enough to call it that). I see far too much synecdoche surrounding programming, and I'm afraid that's not likely to end soon. Some people mistake programming for a science, while others mistake it for pure math, and others mistake a complex macro-powered server configuration for all of computer programming. This confusion needs to be defeated before the tensions surrounding the field can be released. Programming is all of these things, and none of them is all of programming.


        Christopher E. Stith
Re^2: Programming *is* much more than "just writing code".
by BrowserUk (Patriarch) on May 09, 2007 at 18:34 UTC

    It's hard to pick out a appropriate quote from your first six paragraphs, it would require quoting the whole lot I think. I completely agree with you.

    Programming is still yet a very young discipline when compared to most others. It has yet to established itself in the same way or to the same level of scientific reproducibility as the older disciplines. it is still searching for it's equivalent of the universally standard nomenclatures enjoyed by those other disciplines, and that's reflected in the constant strains between existing and new languages; existing and new methodologies; existing and new tool-sets; working practices; et al. Everything within the industry is still in flux.

    In part, this is because unlike the historic, slow boil evolution of the scientific methods involved in mathematics, and the other sciences, which evolved over centuries, through the hands, minds and publishings of a few greats, working primarily alone, but in communication with their (few) peers; programming is evolving in a world of mass communication, widespread, low-level participation, and commercial pressure. It is not so much evolving as constantly revolving. (As in revolution, but also going around in circles. :)

    With each new generation, comes a least one 'new' methodology, language, set of working practices. And usually many of each. Each will be held up by it's proponents as the magic bullet that will revolutionise the industry. The Holy Grail for productivity, or reliability, or re-usability, or efficiency. And usually all of those. Invariably, each of these revolutions has some merits. Equally invariably, they have their flaws also.

    In this ever changing, and multi-frontiered world, it has become nigh impossible to be aware of all the developments as they happen, never mind keep pace with them sufficiently to be able to pick out those parts that will persist and make it into the future. The historical debates between expert practitioners, often conducted over years by letter, or even the publishing of books, has no true equivalent in the modern world. Single ideas that would have formerly been debated over years, (sometimes decades), come and go in fleeting electronic moments. Lost in the noise of internecine wrangling; interpersonal debates; ad hominem attacks; general and specific flame-fests of all kinds. Everyone has an angle; an axe to grind; an ego to defend; a buck to make.

    The software industry spends far too little time exploring and debating the outcome of ideas, methods and practices. Instead of looking back at what was done, and trying to determine what was effective and what was not; how small changes might have made big differences to outcomes; how small parts from several methodologies could be extracted and combined to produce something bigger than the sum of the parts. As an industry, we are always looking for the next magic bullet. The next revolution that will save our collective souls. We discard not just the bad parts of what we did before, but everything, and leap upon the next bandwagon like it was a number 32 bus headed for nirvana.

    The art of debate has been lost. The merit of seeing the other guys point of view; the benefit of constantly re-evaluating ones opinions in the light of the other guys thoughts and experience; is effectively dead within the programming community. If you use Perl, you must also be convinced that CamelCase is bad; must subscribe to the OSS ethic; must be one of the LAMs in LAMP.

    I'm a relatively new study on the history of mathematics, and I am aware that some of the historical greats were far from ego less. I know that there were, at various points in time where two or more of the greats coexisted on this earth, considerable and heated (if you can get heated when communications is by a letter back forth every month or two :), disputes over big ideas. But, from what I read, on those occasions when the owners of those egos and conflicting ideas got together in the same place, they would happily sit down and share a meal and a few bottles of wine before or after a face-to-face debate where they each held opposing views. Technology means that we are able to communicate ever more quickly and with ever wider groups, but the human part of the equation hasn't kept pace. We humans still fall into our historic roles of tribes and combatants.

    To try and bring this back on track. The software industry became an industry before it became a discipline. There are many, too many, languages, methodologies, sets of working practices, existing and coming on stream everyday, for any one person or group of people to be able to say what is best and what is not. Where pure research (into software and development) exists, it tends to be very far removed from the every day realities of commercial practice.

    To badly paraphrase a line from a sci-fi program: the software industry is still young; it has much to learn. Sometimes, the best way to move forward is to look backward.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      BrowserUK wrote:
      I completely agree with you.

      And I found that to be a very puzzling remark at first, because it was pretty clear that MadraghRua was disagreeing with you:

      MadraghRua wrote:

      Going back to you original point, I think what you are advocating is great if it's your own stuff, you are working within a stable group or you are very experienced in all idioms of yor language/development platform. The main problem is the level of documentation needed for projects where the personnel are not stable and the next developer is someone who is not a script kiddie but is not as fluent as the last guy to comprehend and be able to make the modifications necessary to complete a job.

      But I gather that this is the reason you mentioned the "first six paragraphs", because you were explicitly ignoring everything else.

      ("The art of debate"?)

        But I gather that this is the reason you mentioned the "first six paragraphs", because you were explicitly ignoring everything else.

        Yes. That is a fair assessment. As with most of the other contrary points made in the thread, including your own, every point MadraghRua made in his last two paragraphs was addressed in my original post. I saw little purpose in further reiterating my counter arguments to those points, as I saw nothing in those last two paragraphs that in anyway diminished those counter arguments.

        His premise is, that when some relatively expert coder constructs a piece of code, he should make an attempt to review the code he writes in the light of the possibility that the next guy that will view that code may not be as expert in the language being used. He, (the expert original coder), should then add comment cards to explain anything that he feels that the next guy might not understand.

        I addressed this premise in my OP. Nothing in those last two paragraphs forms a counter argument, so there is nothing there to change my mind, and nothing that I can address other than through repetition.

        However, since you seem bent on continuing the discussion, I'll have a go at reformulating the arguments in a way that I hope will lead you, and anyone else still interested enough to go through the exercise, to either produce cogent argument against my point of view, or to accept it.

        What assumption should I make about the level of expertise of the guy that might follow me?

        • Will he understand about context?
        • Will he know when to use $array[ ... ] and when to use @array[ ... ]?
        • Will he consider map too complex?
        • Does he understand the concept of recursion?
        • Will he understand
          if( !$cond1 && !$cond2 && !$cond3 ) { doit() )
          perfectly, but somehow be flummoxed completely by
          doit() unless $cond1 or $cond2 or $cond3;
          ?
        • Will he consider regex some kind of scarey magic?
        • While he consider this clear and maintainable?
          #! perl use strict; my %data; while( my $line = <DATA> ) { chomp $line; my $position = index( $line, ':', ); my $key = substr( $line, 0, $position ); my $value = substr( $line, $position + 1 ); $data{ $key } = $value; } my @unsorted_keys = keys %data; my @sorted_keys = sort @unsorted_keys; for( my $index = 0; $index <= ( scalar( @sorted_keys ) - 1 ); $index++ + ) { my $key = $sorted_keys[ $index ]; my $value = $data{ $key }; print "$key : $value\n"; } __DATA__ a:1 b:2 c:3 d:4 e:5
        • Will he consider this, unmaintainable, deliberately obfuscated, arrogant, 'show off' code:
          #! perl -slw use strict; my %data = map{ /^(.+):(.+)$/; $1 => $2 } <DATA>; print "$_ : $data{ $_ }" for sort keys %data; __DATA__ a:1 b:2 c:3 d:4 e:5

          An interesting exercise for anyone convinced that comments help is to try and add comments to the above two snippets. If you feel like doing that and then posting the commented code, please do, because I will have great fun :)

        Even if it was possible, and desirable, to agree some 'base level' of understanding of Perl syntax below which it was not necessary to explain in comments; it would never be possible to reach a universal agreement at where that barrier should be set.

        And even if that barrier existed, requiring each programmer to come up with accurate, meaningful and useful comments to explain any construct or idiom that breaches that barrier, is simply ludicrous.


        The answer to the problem of encountering code you do not understand, is to educate yourself.

        Look it up or ask someone!

        Take 5 minutes now to experiment with the construct until you do understand it, and then you will understand it forever. Avoiding it forever, because it confused you the first time you encountered it is silly.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://614456]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others chilling in the Monastery: (4)
As of 2024-03-29 00:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found