That sums up many of my feelings quite well.
I've thought seriously about how to bring such professionalism to software development. Rather than try to teach people the "true meaning" of "professional", I talk of making real software engineers, like the engineers that build bridges.
To bring engineering / professional practices to software development will require gradual steps. But I'd like those gradual steps to be made more visible so that people can notice the attempts and some can throw their support behind them.
The basic concept is to start to encourage a small number of the best software developers to make the commitment of becoming true professionals worthy of being called "engineers". These few pioneers would be required to be trained in basic priniciples of software development that are so often neglected, including formal risk assessment, real encryption and security analysis, mathematical evaluation of software quality (such as statistical analysis of testing), and thorough testing methodologies. They would be ethically and legally bound to high standards as described in the article you linked to, in order to provide a "higher law" than "he with the gold makes the rules" that is so common in our industry today (one that finds no shortage of apologists, unfortunately).
The role of these engineers would be to supervise software development, ensuring that proper techniques are used and ethical practices followed. Then market forces can be used to encourage the growth of software engineering. Software projects supervised by an engineer would be "certified" and customers could choose if they wanted to pay the extra price to buy the product that was certified vs. one that isn't. Once we have software engineers, the U.S. federal government would likely quickly come to require engineering certification for much of the software that it buys (especially projects with important safety considerations).
And, it is my experience that I waste a great deal of my time as a software developer in having to deal with the shoddy software that I use. So I suspect that the market may realize that the engineered software can actually be the better bargain.
I need to research how other engineering disciplines came to be. I suspect that at least two universities and two professional societies would have to join forces to create the first approximation to a software engineering program. I'd want accreditation of universities, licensing of engineers, requirements for education, continuing education, ethical standards, etc. all evolving through community review. All of this built up by the universities and societies setting these high standards and then government passing laws to give weight to the results.
I also suspect that this won't happen until some tragedy where a lot of people die because of shoddy software.
As for us peons... I'm shocked that so many software developers so consistently appear completely unwilling to demonstrate their expertise to whoever is paying them for their expertise. The choices aren't "do what the boss / client says even though it's a really stupid idea" vs. "tell the client how stupid they are and probably get fired".
My reaction to "stupid" requirements is to discuss the requirements with those who set them (often taking a bit to find what the real source was) to understand why they think it is a good idea, explain why I, as a hired expert in the field, feel that it isn't the right choice (that is, to give them the benefit of the expertise that they are paying me for), and to help them to understand so that they can make a better decision(though not always a different decision).
Most often, the stupid requirements were not properly evaluated. The first attempt at raising the issue usually gets a "that's the way it is to be" simply because my immediate supervisor didn't make that decision and so isn't the one who can authorize me to change it or because he helped make the decision and so it takes a small amount of explaining to get him to see things from a different angle.
I talk to my supervisor and either he explains it better so I realize why it is a reasonable compromise, he understands my objections and the design is changed, or we realize that he doesn't have the full justification and I'm not just overlooking something which means that I move on to talk to the next person that is closer to the source of the decision.
This sounds complicated, but it usually doesn't take long and it is just good communication that not only demonstrates that I'm taking the job seriously and that I have some expertise, but also almost always improves someone's understanding of the goal and very often improves some aspect of the design. Good communication makes a project more likely to be successful.
My first "real" programming job was for a company of about one dozen employees. At one point management become concerned about people using our software without paying the proper fees to us. They informed us developers that we'd need to write a licensing system that would disable the software if lack of payment was detected. Two of us said it was a bad idea, a really bad idea. We explained that every license server we'd seen has problems, often serious problems, where people had paid but the software refused to work. We noted that people could die (due to the nature of our software) because of these inevitable problems with trying to make the software be able to cripple itself.
Eventually, management responded that they'd heard our concerns but that it was still decided that licensing would be required. We each separately responded that we would refuse to write it on ethical grounds but also because it is would not be in the company's best interest based on our expertise. We wouldn't prevent other developers from writing it, of course. We were told that we'd be fired if we didn't write it. We mostly shrugged.
The project to add licensing never materialized.
Another time I was called in to help write up an official response to a proposal. I didn't understand one of the requirements. I asked about it and it became less clear to me why we'd want that and the others didn't appear to really understand the justification for it. I said we need to go talk to someone and get this clarified. I was told by my boss's boss that this was just the requirement and it was our job to fulfill it, not to question management's decisions. I politely noted that we couldn't do a good job if we didn't properly understand the background and left the meeting to walk to the president's office. I was followed by my boss's boss who was rather upset that I was being so rude and unprofessional.
I explained my concerns to the president and he immediately responded that they hadn't considered that when they made the decision and the decision was changed, after less than 2 minutes of talking, and he thanked me for informing him of the problem.
Just this week I had a rather small but complex feature requirement that was rather precisely if quite simply spelled out in the specification document. Unfortunately, the specification defined the only moderately complex rules that were to be implemented but made absolutely no mention of the justification.
I did not just run off and implement what was spec'd out. It wasn't that I had any objection to the design. It was just that I had no way of validating that some simple mistake hadn't been made in recording the details of this part of the design and not enough ways to validate that I was interpretting the specification correctly.
I spoke to the author of the specification document who said it was a complex decision, something to do with avoiding more expensive delivery options, and then forwarded to me the e-mail chain in which the decision was made. Unfortunately, the e-mail chain showed some give-and-take and evolution on what was decided, but again lacked any elaboration on the motivations.
So I asked my boss about it and got a only few more tidbits. So I interrupted the ever-over-busy principle architect because he had been involved in the e-mail chain. He had some more parts of the justification. I bounced my new-found bits of info off the people I'd already talked to and got a bit more clarification. I did some digging into some rather complex moving parts that were required including talking to some database developers.
In the end, I implemented exactly what was specified in those 4 short sentences of the design. I also included two pages of documentation in the code covering exactly why it was done that way.
Oh, I also uncovered some flaws in the database logic that I fixed and forwarded my extended documentation to the parties involved since none of them had the whole picture. I got one more clarification in response. I think that was a job well done.
By the way, yes, we are all quite busy and under dead-line pressure. And today someone new came across this same feature and was completely baffled at what the justification might be. A whole lot of time was saved because I just pointed them at my documentation. Personally, I'd have included the justifications into the spec document, but that wasn't my call and the existence of the my documentation (checked in with the code) is enough.
Finally, there is another feature that just makes no sense to me. I've mentioned it to several people when the subject came up and they all report that it was discussed to death and that is just how it is. The feature stays the way it is. However, I now know who made the decision and understand some of the history behind it. So, it won't be long before I casually bring up the subject with the VP and founder (this is a much bigger company) so I can likely convince him that the decision used to make sense but doesn't anymore. And the product will eventually be improved because of this and the company will benefit in some tiny way...
That's my job.
