Re^2: Programming *is* more than "just writing code" ( but it's *not* a science) (was: Programming is more than writing code, but it's *not* a science)

[mr_mischief]

There is a serious flaw in comparing programming to scientific research. Programming in the large is not science.

To equate programming in general to fields of science is like equating architecture and construction of a building to physics. Programming uses CS as a basis the way architecture and construction use physics as a basis.

There are a couple of simple rules at work here that often get overlooked in the programming realm: The size and complexity of the project determines how sophisticated and informed your methods need to be. Also, the more levels of knowledge being built upon your contributions, the more fundamental and universal your contributions need to be.

Let's face it. A cook is a chemist of sorts, and so is a candle maker. Neither is going to rewrite textbooks about chemistry. A carpenter or bricklayer has some sort of physics knowledge, but they are not going to split the atom or need a particle accelerator. At one time, a cab driver or a cashier actually had to know some math, but they weren't in the business of proving the number of dimensions in the universe.

We don't have any solid, universally accepted differences in title between a sysadmin who's really talented with shell scripts and someone who writes frameworks for generalizing facial recognition software. Just because both might be called "programmer" or "software developer" does not mean the two are equivalent. I think this is a large part of the confusion in the software industry. I also think it's part of what causes so much friction between the highly qualified, very method-oriented people doing lofty things and the people who write small applications for small clients in small markets.

The truth is, you don't want everyone to be a scientist. You don't even want everyone to be an engineer. You won't wait for a Pasteur or a Salk to mix your cough syrup. You won't wait for a Hawking or an Einstein to build your house, or for a Tesla or Westinghouse to wire it. What you want, down in the trenches, is people who take the results of research and engineering and figure out how to apply those components and best practices to the project at hand. Researching fundamentals is important, but so is having homes, food, clothing, and a community site like Perlmonks to debate the point. If the research and experiments have already given us evidence that a particular way of doing things is a good way, why can't some people just use those good ways while others further the research?

Just because some people don't use best practices when they should doesn't mean everyone should be in the realm of developing new fundamental knowledge. That's where the line between technology and science lies. Technology benefits from science, but people as a whole benefit from the fruits of technology more than directly from the fruits of science.

Day-to-day programming is, like building construction, more a technological trade than a science or research topic. It's a method of combining established parts using established methods to further one of a set of fairly common goals. Sometimes a building contractor finds an innovative and truly better way to build part of a house. An architect is more likely than the contractor to find a new and better way to design the whole house. That same architect might develop a new and better way to fasten two materials together, but an engineer is more likely to do so. The engineer might develop an altogether better material from which to make the fasteners, but a researcher is more likely to do that. It's this way in any field, really. I'm not sure why people get the idea that software should be different.

Another trend you'll notice if you look at more established technologies is that these mostly stratified layers become more established the longer a trade is around. We didn't used to have a designer, a spinner, a dyer, a weaver, a cutter, and a seam sewer for making garments. People used to design and build a car single-handedly, but now there are auto researchers, auto engineers, auto factory engineers, auto factory employees, and auto factory robots. In time, I think we'll see such lines drawn in the software industry, of course with some mobility among layers. I just hope the titles aren't still, 'chief software designer', 'system analyst 3', and such meaningless chaff.

[Joost]

Excellent post. :-)

I always liked the comparison between software and building construction. I think it's one of the best "real-world" analogies we can use to explain what we do, but there are differences.

I view myself as being somewhere between an engineer and an architect. I like to "think with my hands" so to speak: think up a rough overall design, implement it and then move code around, rewrite parts until it works well and makes sense.

The reason that works well (or at all) in software development is because of a pretty fundamental difference between building construction and software development; there is hardly any difference in software between the design and the product. It's only a matter of abstraction. Any really comprehensive software design is practically code (and you can argue that the reverse is also true). That's also a reason I don't like/believe in "automatic code generation from designs" - if your design is that comprehensive you're still programming, only chances are you're not using the right tools for the job.

[tye]

I'd just like to add that this high degree of similarity between a sufficiently detailed design and a sufficiently factored implementation was not nearly so much the case those few decades ago when I started programming.

Languages and programming techniques have been moving toward allowing you to write a program by writing more design-like code while design techniques have been encompassing more of the details that are required when you write the code. There has also been more development of thorough methodologies that encompass both design and coding and these produce even more similarity in design and code when used.

When I've done design that really pushes my abilities, I've longed for a tool that would let me build the framework by typing up some quick class definitions, change panes to see a graphical representation of the resulting relationships, make some changes in that GUI pane, switch back to the first pane and see the classes reflecting the results of my mousing, and continue back and forth like that. I've used some tools that try to come close to that, but none of them did it well enough to be a big pay-off. I always ended up avoiding much of the "help", using them in a "less integrated" manner, and being a bit sad that they were still too awkward or ridgid or whatever such that they became more of a hinderance than a help when I tried to make full use of them.

Note that I agree with your final statement. Even if I had the perfect tool above, I'd only be using that for the design part of the coding. There'd be a line beyond which I'd then be coding and I'd stop using the design tool. Looking back, I'm a bit surprised to realize that C++ appears to me to have done a pretty good (but not perfect) job at effectively separating the design from the code where *.h is nearly all "design" and needn't contain hardly any "code" while *.cpp contains very little "design" (mostly just restatements of chunks of the design that is in the related *.h, and somewhat annoyingly verbose restatements at that).

I do wish C++ allowed the "private:" bits to not be specified in the *.h, however. I also found that enforcing some rules about the order in which you write the different parts of your C++ class definition made it much easier to see the "design" when looking at the *.h. And this all makes me realize that I've not noticed this design/implementation break nearly so much in other languages. And I'll end my musing there (bringing the topic somewhat back to Perl and even to Perl 6, if one is paying proper attention). :)

[Joost]

Languages and programming techniques have been moving toward allowing you to write a program by writing more design-like code while design techniques have been encompassing more of the details that are required when you write the code.

That's true. I was going to make some kind of comment on that but it was late and I couldn't write it down right :-)

In any case, I'd prefer to have a language that makes it easy to define classes etc, a tool that makes it easy to refactor and some way of visualizing hierarchies etc directly from the code.

My experience with design tools always made me wish I could just type the class interface instead of having to open, drag, click, select and type my way around just to add a method.

[mr_mischief]

Thanks for the support.

Like all symbolic language, there are inconsistencies in this metaphor. However, I find that sometimes a look outside the immediate surroundings of the topic is necessary. There's a psychological trick here in that removing one's thoughts from the topic at hand for a moment can help one focus more clearly once the mind returns to that track (train metaphor -- watch out). The fact that some semblance can be drawn helps serve the point, but it's far from the only reason for the comparison. I could have just as readily talked about how cute my step-nephew is for a paragraph before moving on to the topic, but I thought the metaphor could serve dual purposes.

It's true that at some point comprehensive software design is almost code and that abstract code is nearly a design. I think there's another point that separates building construction and software development even more, though.

I find that the biggest difference from building construction is because there's so little cost to moving code around. In building construction, it would be wonderful to pull a wall out here and put it back in over there. Unfortunately, there's a lot of physical labor involved, the materials are usually damaged during the remodeling, and the structure may not be as sound afterward. With software, it's much quicker and easier to move whole chunks of code from one place to another, change the interfaces, and generally muck about.

This is in large part because software is abstract, but not necessarily because it is close to the same level of abstraction as the plan. Actually, I'd bet lots of architects are using the 3D building design software on the market to do with virtual copies of buildings what we do with code. Think of it as a visual integrated development environment using pluggable modules to create a value-added design by pointing and clicking, with certain details left to hand-written code(s)...

Now that I've almost choked reading my last buzzword-riddled sentence, let me say that we'll probably never find a metaphor that fits programming better than devising a meal recipe. There are many ways to compare parts of software development to parts of other ventures, but it is so fundamentally different from everything that came before that nothing is particularly adequate in every way. Technical writing can be almost programming, and hardware design can be exactly programming, so those don't serve well as metaphors. A metaphor has to be enough alike while also enough different, and that's a fine line in any case.

Strict research science is still not a preferred metaphor for software development (as a whole) in my mind. Part of the field is certainly science (or close enough to call it that). I see far too much synecdoche surrounding programming, and I'm afraid that's not likely to end soon. Some people mistake programming for a science, while others mistake it for pure math, and others mistake a complex macro-powered server configuration for all of computer programming. This confusion needs to be defeated before the tensions surrounding the field can be released. Programming is all of these things, and none of them is all of programming.