Re^9: Beyond Agile: Subsidiarity as a Team and Software Design Principle

[einhverfr]

Machines don't make decisions; software does.

Define "decision." I don't see a principled line between the sorts of decisions that software can make and those a physical machine can make. Consider for example, the old electromechanical phone switches....

Machines do not adapt to their inputs; software does.

Again I am not sure what this means. Maybe it could use more explanation?

Machines do not prioritise their responses to inputs; software does.

Why not? Certainly I could imagine a machine which would prioritize processing of inputs.

Machines have physical limitations; software has only logical limitations.

I think I get what you are saying here but I am not sure what relevance it has. Certainly the system running the software has physical limits. So software is a sort of abstraction and being an abstraction allows us to think about it disregarding the limits (I think that's dangerous though). But I think there are also human limits to software and because software is an abstraction we don't tend to talk about those.

Bottom line: Software is to hardware as pheromones are to ants.

That's actually a good analogy. I like it. But pheramones are subject to physical limits (temperature, air characteristics) etc. right? ;-)

[BrowserUk]

1. Machines don't make decisions; software does: ... Consider for example, the old electromechanical phone switches....

   The only decision being made there is by the human being that dials the numbers. They dial 9; nine pulses are generated by the dial; those nine pulses cause the switch to rotate 9 places.

2. Machines do not adapt to their inputs; software does: Again I am not sure what this means.

   If the pilot of a pre-fly-by-wire airliner dies at the stick and slumps forward on that stick, the plane will inevitably nosedive.

   With fly-by-wire airliners, the flight control software can and does intervene and disregard pilot inputs that take the aircraft outside of its safe flight envelope.

3. Machines do not prioritise their responses to inputs; software does: Why not? Certainly I could imagine a machine which would prioritize processing of inputs.

   Then I invite you to publish your imaginings of a purely mechanical system for applying the brakes on a car if it detects a human being in the vehicles path.

4. Machines have physical limitations; software has only logical limitations: I think I get what you are saying here but I am not sure what relevance it has.

   From Nancy G. Leveson. Aeronautics and Astronautics.MIT:

   Software is pure design without any physical realization and therefore “fails” only by containing systematic design defects. In fact, software can be thought of as design abstracted away from its physical representation, that is, software (when separated from the hardware on which it is executed) is pure design without any physical realization of that design. While this abstraction reduces many physical limits in design and thus allows exciting new features and functions to be incorporated into spacecraft that could not be achieved using hardware alone, it also greatly increases potential complexity and changes the types of failure modes. With respect to fault tolerance, potentially unsafe software behavior always stems from pure design defects so redundancy—which simply duplicates the design errors—is not effective. While computer hardware reliability can depend on redundancy, dealing with software errors must be accomplished in other ways.

   The recent SpaceX Falcon 9 mid-air explosion was caused by the failure of 2ft long, 1" diameter steel strut. Rated for 10,000lbs, it failed at 2,000lbs. The fix is relatively simple; redesign the strut to increase its strength and test each one individually before flight. It took them 3 weeks to find that physical error.

   Contrast with the life, path and root cause of a software error:

   There was, however, a potentially serious software error that occurred in April 2009, just two years before the Shuttle’s retirement. The error manifested itself in flight STS-126 a few minutes after Endeavor reached orbit. Mission Control noticed that the Shuttle did not automatically transfer two communication processes from launch to orbit configuration mode. Mission Control could not fix the problem during the flight so they manually operated necessary transfers for the remainder of the flight. The pathway for this bug had been introduced originally in a change made in 1989 with a warning inserted in the code about the potential for that change to lead to misalignment of code in the COMPOOL. As more changes were made, the warning got moved to a place where it was unlikely to be seen by programmers changing the code. The original change violated the programming standards, but that standard was unclear and nobody checked that it was enforced in that case. Avoiding the specific error that was made was considered “good practice,” but it was not formally documented and there were no items in the review checklist to detect it. The SPF did not identify the problem either—testers would have needed to take extra steps to detect it. The SAIL could have tested the communication switch but it was not identified as an essential test for that launch. Testing at the SAIL did uncover what hindsight indicated were clear problems of the communication handover problem, but the test team misinterpreted what happened during test—they thought it was an artifact of lab setup issues—and no error reports were filed.

Comparing software to a machine at anything more than a totally superficial level, completely misrepresents the nature and complexity of software.

This is a complex machine, but it is roughly equivalent to 10 lines of Perl.

[einhverfr]

I guess what I am not sure about is whether those are really differences in kind or just degree. Let's take for example a WWII-era torpedo. It is effectively programmed mechanically without software. It cruises at a given depth and under certain inputs decides to explode. Similarly unmanned cruise missiles have been around with similar criteria longer than we have thought about these things in terms of software.

It is true that software allows these decision trees to be more complex than one gets with purely mechanical systems but prioritizing inputs is a basic strategy in safety engineering. You might have a riding lawn mower disengage the blades when the rider stands up for example. On the other hand, if the pilots die in a modern fly by wire airplane, it will still eventually crash just later (see Helios Airways 522). To a certain extent you have significant issues with human troubleshooting and highly automated systems (see this article in IEEE Spectrum) because of complexity but I am not sure how software makes that significantly worse directly other than just increasing complexity.

The software you speak of is an abstraction that doesn't exist in the real world. A more realistic view of software is that it provides a mechanism to adapt machines to changing needs. If we are going to disregard speed there's no real reason that electro-mechanical systems could not do anything electronic systems can, just slower and if speed is a criteria you are limited to the physical requirements of the systems today.

This is also where my disagreement with the guy promoting that book is, namely that software exists in a context, teams don't just go home (someone has to maintain and manage the machine). Software in the real world doesn't just run itself.

[BrowserUk]

It is true that software allows these decision trees to be more complex than one gets with purely mechanical systems but prioritizing inputs is a basic strategy in safety engineering

From my perspective, you've made my argument for me with that statement.

WWII torpedos (non-wire guided) used a gyroscope to get them to run in a straight line. The trick to getting them to hit the target was to launch them on the right course. Calculating that course was a matter of solving an equation, which was done using either a slide rule, or later and electro-mechanical computer, that "required two extra crewmen: one as an expert in its maintenance, the other as its actual operator.".

That same equation could again be solved with 10 or 20 lines of Perl. The job of that electro-mechanical computer could today be performed by the same disposable chips that you find in musical greetings cards for a couple of quid.

A modern torpedo "can also use their own active or passive sensors to execute programmed target search, acquisition, and attack procedures. The torpedo is designed to detonate under the keel of a surface ship, breaking the ship's back and destroying its structural integrity. In the event of a miss, it can circle back for another attempt.".

It simply wouldn't be possible to construct a purely mechanical system equivalent to this; but if is was, the torpedo would need to be the size of nuclear sub in order to house the all the gears and motors; and probably still wouldn't have room for a warhead.

And that's it. Even the simplest of software systems contains the equivalent of millions upon millions of mechanical parts.

By the time you get to something like the browser you are using to read this with its ability to be programmed in multiple different languages -- html/xml/css/javascript -- display and process images, video, vector graphics; communications; encryption/decryption; etc. etc. It is more complex than even the most complex machines. If you add in all the subsystems of the operating system; communications stacks; device drivers; etc. required to allow it to run, you've upped that complexity by another order of magnitude or two. Factor in all the ways and timings those different subsystems interact and you've upped the ante again.

If you're going to compare software to hardware, you shouldn't compare it to "a machine"; but rather something like a large car or aircraft production plant; where you have 1000s of machines all interacting and working in concert. Only then do you begin to approach the same levels of complexity as even a fairly modest software system. And for the largest software systems you need to be thinking in terms of something like all the machines and systems in a large town or small city.

But complexity is only one of the 4 major differences between software and hardware I listed above.

Another that I didn't get to: these days when designing a building; or a ship; or a bridge; or a car or aircraft; or any other complex mechanical engineering project; we use computers to do it. We protect entire buildings/ships/planes in transparent 3D images that we can walk around, zoom in, check for clearances and paths; even light levels, acoustics and aesthetics.

Despite that we use computers to write software -- editors, compilers, VCS etc. -- we still essentially write code by hand; and write the code that tests the code by hand. We don't have the equivalent of CAD/CAM or even the equivalent of simple gauges that can quickly, accurately, repeatedly and definitively test a mechanical engineers work.

We've succeeded in working out how to apply software to assist in doing (or even do) almost every other type of work; but we've yet to even begin to apply it to the task of constructing software. I think there is a good reason for that; and it isn't because were protecting our jobs.