|Problems? Is your data what you think it is?|
Why? - threefold: why does it have to, why should it, and how could it?
There are still Master Furniture makers like Joseph Hemingway around, and most of us would love to be surrounded by the type of furniture he produces, but few people can afford to be so.
Software production costs way too much. The development costs are alleviated somewhat by the cheapness of mass distribution, but still it will have to get cheaper.
For the how, you need to look at better tools. In particular, a set of standards covering the interfaces for reusable, interchangable components (libraries) so that we programmers can bolt together more parts of our programs from them, more quickly and with less interfacing glue.
And there will have to be less languages. When I was growing up there were a dozen or more mass production car manufacturers in the UK, a dozen or so in France more in Germany. Now there are less than a dozen across Europe. And many of those are brand names owned by the same parent conglomorates. (Nissan == Renault; Volvo/Jaguar == Ford; Seat/Bentley/Bugatti == VW-Audi; Gone are Austin; Morris; Humber; Hillman; Roots; Vanden Plas; DAF; and many more.)
Consolidation is inevitable. As is automation. People have been trying to automate program generation since programming began. There was one pretty serious attempt made way back in the early '80s called The Last One. (Meaning the last program that would ever have to be written by hand.)
We know that it got no where, but it won't stop people trying. And eventually succeeding. There is nothing more suited to applying the power of computers to than the creation of programs. We programmers are applying the power of computers to ever more complex fields of endevour every day, but still insist on hand crafting our own endevours using a disconnected, non-interoperable set of tools that go back decades. That's the equivalent of corporate accountants and commodity traders still doing double entry book keeping using quills and ledgers.
C & Fortran compilers can already generate better optimised assembly code in the general case than an experienced assembler programmer can achieve. The Haskell compiler can basically take a mathematical description of an algorithm and turn it into extremely fast and robust code.
But most of use are still coding in a general purpose editor, then submitting our code to the compiler/interpreter for syntax checking. Then stuffing that into a text-driven source control database. And recompiling/interpreting the source code from text representation every time.
Sure, there are IDEs that will syntax check (some) languages as we type--but they are still widely eshewed. And we have object libraries, but they are in a different format and once created basically become static entities. Some compilers support pre-compiled headers and incremental compilation, but their databases are in proprietary formats and can't interoperate with each other, or other tools.
All of that has to change. And it will. Programmers have to start applying their art to their own works and tools. It's inevitable.
For more reading of (my writings) on this subject see also: Re: How do you view programming and Re+: (OT) Programming as a craft (loong post. Even by my standards!). There is more kicking around this place, but that is probably more than anyone will read.
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.