|The stupid question is the question not asked|
This will be heated, but I really do not intend it to be a flame. However those who like to skip heated discussions should skip this post. If you think that heated discussions are always inappropriate for the Monastery, please drop a -- in the bucket first. But I actually think this is going somewhere. I believe this is constructive.
First of all I don't care whether you disagree with me. I am giving you my professional judgement of what the facts are. Take or disregard as you will. But as far as you disagree with my rant, I believe you either are mistaken or are missing key details.
For instance a case where you are out and out wrong is that repetitive code is sometimes necessary for efficiency. If you really need the efficiency of repetitive code, then preprocess your code to autogenerate the repetitive stuff. But the code that you maintain should avoid having that kind of fragility.
For instance a key detail you miss is that I work in a small company. While there is considerable differentiating of what we do with data, there is little differentiation between what data we work with and what access we need. Therefore everything you said about authentication and permissions is completely irrelevant to me.
Now I admit it. I find dealing with you very, very frustrating. You are obviously not a stupid person. You go out, put a lot of work in, learn about a lot of stuff. But you really seem to have no appreciation for how to rate what you learn, what principles to pay attention to, what goals you can aim for. I covered part of this in Re (tilly) 1: Topics in Perl Programming: Table-Mutation Tolerant Database Fetches with DBI.
Perhaps I can summarize it by saying that you appear to be motivated by the desire to accomplish neato things in a gee whiz manner. That may be fun, but that isn't what software engineering is about. What software engineering is about is managing complexity. That does not mean avoiding complexity, although it is good to avoid unneeded complexity. That means that as programmers we are asked to accomplish complex tasks, and over time we will be asked to both build on what we did, and modify it often at the same time! This is inherently hard and inherently leads to confusion.
The aim of software engineering is to make dealing with these necessarily complex situations a managable task. Being straightforward is good. Being straightforward to the point of having an unwieldy amount of code that has to be maintained just so is bad. Hiding information under an interface is good. Having interfaces multiply like rabbits so that nobody can ever learn the system is bad. Not reinventing wheels is generally a good idea. But avoiding doing so when the wheel doesn't work, is inappropriate, or leads to the above issue about having too many components is bad. Having idioms, ways of saying things that are instantly recognizable and allow people to "chunk" code to higher levels is good. Turning your idioms into cut and pasted mistakes scattered through your code is bad. Having policies and procedures and change control is good. Having the paperwork reach a point where you do not get the job done is bad.
Every good thing you can do has a corresponding cost. The key is to understand both the benefits and the costs so that you can evaluate situations, understand the trade-offs, then find and implement an appropriate solution. That is what we are paid to do. We have jobs because somebody thinks we can be trusted to do that, and we should try to do it well.
However you have not a hope of succeeding unless you understand that that is the goal, it is the point. It is what is important.
Now you are not stupid. You know that, I know that. I honestly believe that you are capable of staying aware of this principle, and paying attention to the trade-offs. Apply some of that desire to go off and learn lots of stuff, go read some stuff by Steve McConnell. But most importantly, just constantly try to evaluate how every tool and idea you run across helps and hinders (they all do both) the ability to manage the inherent complexity that we have to work with. Think less about what you want to do, and how you are going to do it. Think about why you are doing things, and think through ways it can go wrong.
I really believe that you can do that. I would love to see you do that. I promise you that if you do do that, you will improve. You will develop a sense of what things are design mistakes and what are good ideas. You will develop an ability to anticipate what is going to lead to trouble and head it off. You will be able to write code that isn't painful for people like chromatic and myself to read. Some of these things will happen faster than others.
But until you do that you will continue to crank out code, designs, and questions that make good programmers wince. And without you knowing how to judge good from bad, we will continue having trouble explaining to you the whys and wherefores of where we disagree with you.