Beefy Boxes and Bandwidth Generously Provided by pair Networks Frank
Welcome to the Monastery
 
PerlMonks  

The future of software design

by bprew (Monk)
on Oct 18, 2002 at 04:11 UTC ( #206214=perlmeditation: print w/ replies, xml ) Need Help??

AKA is programming headed the way of auto-mechanics or civil engineering?

By this, I mean to draw parallels between software development and the fields of auto-mechanics and civil engineering. Both currently and in the future.

What do I mean by civil engineering?

By civil engineering I mean a market that requires all employees to be highly trained or have spent alot of time training to be a civil engineer.

What do I mean by auto-mechanic?

By auto-mechanic I mean a market that requires many more employees then civil engineering. Also, one can become proficient as an auto-mechanic with much less training then required for a civil engineering degree.

This also brings up the issue of what each market does. Cars are considered along the same line as appliances, not so cheap, but plentiful. Buildings, bridges and other civil engineering projects tend to be, very complex and sparse in comparison to cars.

The current state of software development

--as seen through my (acolyte)eyes

Currently, I feel that sofware development is a complex field that requires specialized knowledge of computers and software. People can and do teach themselves how to program, but I would bet most programmers have some sort of continuining education. Therefore, the software development currently parallels civil engineering moreso then auto-mechanics.

The future of software development

--once again, largely based on my opinions

I feel that computers will quickly parallel the widespread usage of the car. In doing so, programing languages will continue to evolve to a higher and higher level, especially considering the falling cost of processing power. Eventually, it will mirror the automotive industy. A few (comparitevly) auto engineers designing cars and a large number of auto mecanics repairing or tinkering with cars. Most programmers will be less educated and more self-taught. This will drive salaries down for most, save the designers.

Summary

mostly a way to preserve and furthur my thoughts

How accurate do you think my statements are? Where do you feel software development will head towards? Just imagine the point we get to when everyone will be able to design software and not be limited by the computer :). -- Ben

Comment on The future of software design
Re: The future of software design
by Zaxo (Archbishop) on Oct 18, 2002 at 05:00 UTC

    I think that the fundamental differences between software vs. iron and concrete make your analogy weak, but not irrelevant. Making and maintaining physical things requires a lot of capitalization and a steady predictable market. There are standards of engineering practice in the construction of durable objects.

    The more etherial nature of software, seen in its mutability and cheapness to duplicate, makes the skill of the programmer more important to the cost structure. The speed of change prevents the formation of rigid professional standards. Programmers are valued for their ability to get a job done which has never been done before. Once an application has been implemented a few times, de-facto standards arise for that application. Those standards are broken up and applied to new problems, not as standards but as guidelines.

    The software industry is already broken into divisions like you describe: design, production, consulting, maintainance, support, with similarly differentiated pay scales. The difference is that software at any level needs an aptitude for abstraction, a grasp of mathematical logic, and a kind of linguistic sensibility. Mechanics and engineers have fundamental differences between them, and they each tend to regard the other as the enemy. That situation can arise in software, when warm bodies are expected to maintain what they don't understand, but the fundamental differences are not there.

    After Compline,
    Zaxo

      As someone whose degree is in Civil Engineering (UVA 1993) but programs Perl for a large part of my living, My opinion is in line with Zaxo's: The differences between CS and a 'more mature' disipline like CE are more instructive than the similarities. A fundamental difference is that to be a 'real' civil engineer, one must be licensed. Almost all work done by the profession must be certified by a licensed professional. That certification or 'stamp' carries with it extreme liability (google "joint and serveral")that is mitigated through (expensive) insurance. Also, licensing itself entails 3 year of progressive experience under the direction of a licensed engineer. Licensing is done by the state but is none the less controlled by the profession itself, much as it is with doctors or lawyers.

      Having said all that, I don't think that the concrete and iron aspect of CE is what produces the diffences. In fact CE (like all engineering) is more about information. Deliverables are plans, specs, and designs.

      There is a school of thought that espouses the idea that some fields are professions. These include: Doctors, Lawyers, Teachers, Engineers. Professions have in common a duty to the public and the ability to self-manage due to the unique set of skills that are required.

      Whether or not this pre-20th century idea of a profession can or should embody software development is an interesting question that I have pondered without coming to a firm conclusion. I am certainly not of the belief that all SW developers need be licensed like doctors.

      drat

      -----------------------------------

      "That is the best engineering, not which makes the most splendid, or even the most perfect, work, but that which makes a work that answers the purpose well, at the least cost"
      --Ashbel Welch, President American Society of Civil Engineers, 1882
        And most of those professions have parallel professions which do some of the "junior" work, i.e. nurses, paralegals, etc. So there is no need for all software developers to be licensed at the same level.
Computer Science is dead! Long live Computer Science!
by mojotoad (Monsignor) on Oct 18, 2002 at 06:19 UTC
    I can see where you're coming from with your post; however I don't feel the analogies are a great match. I think you are aiming at correlating the design aspects of programming with civil engineering and the implementation/maintenance aspects with the art of the auto mechanic.

    Programming involves both. Implementation might get easier as paradigms are established, but new problems always surface. Design might not have reached steady state yet (or saturation), like civil engineering. (CE is still evolving, but the rate of new problems has diminished.)

    The fringes of Computer Science share aspects with such diverse fields as Cognitive Science, Psychology, Biology, Philosophy, and yes, Engineering (electrical, mechanical, chemical, etc). In this sense, I don't see CS slowing down at all any time soon.

    Well. I'm leaving aside the prospect of the Singularity, when our own computers and programs began to write themselves. CS will still be evolving even if we are no longer the principle designers.

    Design, as such, leaves behind a wake. This wake becomes more solidified as paradigms and standards emerge as accepted solutions to particular problem spaces. This does not mean the boat has stopped.

    Matt

Re: The future of software design
by dreadpiratepeter (Priest) on Oct 18, 2002 at 15:41 UTC
    I've always make the analogy that Programming is a craft, like Blacksmithing or Carpentry. A craft is somewhere between an art and a science. It has both subjective and objective parts. If it were a science we would already have written a program to do it and if it were an art all solutions would be equally good.
    Because it is a craft, the entry requirements for Programming are a lot lower than those of a science. (Any fool can try to build a dresser, but you need years of specialized training to even consider gene-splicing). Combine this with the extremely low cost of raw materials for Programming and you have the grounds for millions of amateur programmers.
    But, as a craft there is a huge gap between bad, competent and masterful. The difference in quality, reliability and beauty between the dresser I build and the dresser that Norm Abrams (of This Old House fame) builds will be staggering. It is the same with code. My code (I have formal and informal training and 20 years of experience) is likely to bhe infinately more reliable, elegant, and robust then the guy who just finished '21 days to Perl Programming'.
    Or, even the guy who just got his degree in comp. sci., because a craft takes years of practice with a wide variety of projects and continuous exposure to new problems, new tools, and new environments. It cannot be learned in a classroom, it is not a matter of memorization. Good dressers come from years of training and practice. Good code is the same.
    That is why most crafts developed with an apprentice system. To train tomorrow's smiths, you started today. With young people willing (or forced) to spend years learning the craft through application and experience not just through lecturing and memorization. And, to finally come to a point, that is what is missing from the craft of Programming, the apprentice stage. A company wouldn't hire an apprentice Carpenter for an important project, it shouldn't hire an apprentice programmer- for the lead roles. A master programmer with apprentice programmers under her is what is needed. But, here is the rub. A company usually doesn't have the means to separate the apprentices from the masters. Resumes and interviews can be faked. Skill sets and experiences can be enhanced, failures and shortcomings can be glossed over.
    So how do we fix it? The same way the over crafts did. certification. Real certification, based on both knowledge and experience. And not in a specific technology, in a wide range of technologies. Something in an Apprentice, Journeyman, Master vein. And make the certifications mean something. 1 in 10000 programmers might be a Master. Companies with sense can hire using these criteria, and get what they pay for, and a schmuck with a few books under his belt can't pass himself off as a master programmer.
    Boy did I ramble,

    -pete
    "Worry is like a rocking chair. It gives you something to do, but it doesn't get you anywhere."
Re: The future of software design
by Popcorn Dave (Abbot) on Oct 18, 2002 at 17:37 UTC
    I think with your mechanic analogy, you can look at as any idiot can plug something in to a computer to see what's going wrong. So you're always going to have people that can put pre-done bits of code together to make a program.

    However, as with any profession, there are always going to be those who are the craftsmen. The fellow that does the Woodwright's Workshop on public television here in the states springs to mind. You've got someone using primitive tools to produce items in wood. If you gave those to the average Joe or Jane on the street, you'd get sawdust.

    So there's always going to be a need for the craftsmen whether it's engineering or programming.

    As far as anyone being able to design software, don't we have that now with things such as Visual Basic? I know Turbo Pascal played a big part in that perhaps 10-15 years ago. But just by the amount of shareware that's available, and that was even available 10-15 years ago shows that's already happened. Somehow I doubt people are going to spend hours upon hours doing assembly code to produce a program that they *hope* to get someone to pay $10-20 for.

    There is no emoticon for what I'm feeling now.

Re: The future of software design
by blssu (Pilgrim) on Oct 18, 2002 at 18:28 UTC

    If programming was going to turn into something like civil engineering or auto repair, it would have already. The computer industry is 50 years old.

    Programming is not just about "design" or "repair" work. Sometimes it's more like archaeology or tool and die making. We're dealing with complex artifacts built by humans -- sometimes for completely unknown reasons. Frequently we're not powerful enough to solve problems directly so we must invent new tools.

    Programming jobs also blur into many different fields. Industrial design, writing and economics are pieces of tasks that programmers usually end up doing themselves.

    Anyways, the future is going to be a lot like today until a machine-mind interface appears (call the result a "Spiritual Machine" if you're a Kurzweil fan). I have no idea what will happen, but I expect programming will be the first field to find out.

      Anyways, the future is going to be a lot like today until a machine-mind interface appears (call the result a "Spiritual Machine" if you're a Kurzweil fan). I have no idea what will happen, but I expect programming will be the first field to find out.

      I don't believe much in such direct machine-mind interface "added" to adults and that would not be ethical to mess with babies.

      We already have interfaces with the extern world: sensors and effectors. If they are not connected at birth and appropriately trained, they will not be effective later. A blind person at birth that recovers later cannot make sense of the new deluge of information.

      What we do currently makes a lot of sense, we create extensions of our existing senses and effectors (displays, cars...). We have a better understanding of this interfaces to the world than about our brain internal working.

      This idea of machine-mind interface is implicitely based on the (probably false) assumption of an existing brain "API". As I said about existing human effector and sensor, this API exists as a potentiality but fully develop only if we use these sensors and effectors.

      About thought formalized thru language, things are even more complicated. How could we hook directly to something we don't know anything substantial about.

      A good (but not recent) book about language and learning is "theorie du langage et de l'apprentissage" which is the report of meetings between Chomsky, Piaget, Papert and many other luminaries. I don't know if there is a English edition though.

      -- stefp -- check out Nemo

        I agree that connecting sensors to humans would not be very beneficial. Human senses are exquisite.

        What I really need is a few terabytes of error-free, short-term memory! That was the machine-mind interface I was thinking of. It would definitely make me a better programmer. Unfortunately, programs built by people with augmented memory would probably not be maintainable by normal humans.

      "If programming was going to turn into something like civil engineering or auto repair, it would have already. The computer industry is 50 years old."

      Yeah, sure -- we've been making buildings for thousands of years, but a formal civil engineering practice is only a (relatively) recent innovation. Previously builders just sort of did their best and learned from others mistakes. A lot like how programming (currently) is.

      I think it's _way_ too early to see where computer programming is going to end up. Neither of the analogies is particularly apt, since civil engineering is essentially applying scientific methods to well-understood problems. A particular steel beam might have a particular tensile strength, weight, and other well-known properties. Likewise a wooden beam has "well known" properties. The decision of which to use is based on mathmatical formulas balancing cost vs. risk.

      Likewise with the auto-mechanic analogy -- you can become an auto mechanic with little or no formal training, simply learning on the job. Unfortunately you also don't need an IQ much above 80 either. In any event, auto mechanics don't build cars or design them -- professional engineers do that.

      I tend to agree with another poster that pushed a craftsman analogy. Although craftsmen are basically seat-of-the-pants engineers, they do make things (unlike mechanics), and their work is as much art as science (much like programming is right now). Blacksmiths might not know the precise tensile strength of a particular metal bar, but they can experiment a bit and probably find something that will work.

      I think that this 'craftsmen' approach to programming will disappear, replaced with an 'engineer' approach, but only as the problem domains become well-known, predictable, and measurable. This might take hundreds of years, by which time human 'computer engineers' will probably be replaced with intelligent systems, and programming will be like talking to an assistant is now.

        Modern societies have big egos. Were the Great Pyramids one of the early mistakes? ;)

        Programmers have a bit of power now. If we wanted to, we could easily create formal social/professional structures. Do you want them? I don't. Who does? Will your government create them? Would you oppose that? I would.

Re: The future of software design
by YuckFoo (Abbot) on Oct 19, 2002 at 16:05 UTC
    Well, ma'am, the problem appears to be in your Foobar.pm module. Seems there was a useless use of map in void context. We see a lot of this in the late '90s programs.

    We also noticed your HTML-parsing regex is about on it's last legs. I can't tell ya fer sure it'll break next week or even next month, but it's gonna be trouble. Now we could fix this with an HTML::Parser module but we don't have any in stock. If ya like we can order one from CPAN for ya. No charge for the part, but a couple hours labor will run ya about $92.50. That's assuming your HTML::Entities is in good condition. If we need to upgrade, it'll cost you a little more. We just aren't gonna know fer sure until we get the thing into vi and have a look.

    While we're at, you want us to check your taints? We've got a special this month on taint checking and use warnings, only $29.95.

    It'll be about four or five days before I can have one of my boys look at it. We're up to our necks in broke guestbook scripts. Seems Matt just released a new model.

    Hand me your passwords and we'll call you when it's ready.

    Y'all have a nice day now, ya hear.

    FixitFoo

Re: The future of software design
by Felonious (Chaplain) on Oct 19, 2002 at 17:08 UTC
    I don't think programmers can be easily lumped into the analogies of mechanics and civil-engineers, but perhaps programming as a field is analogous to the field of construction.

    Small projects, like adding on to your back deck, can be tackled by do-it-yourself-ers with a little guidance or alot of common sense. Plans can be drawn up in a evening and the job done in a weekend. As projects get larger, the job gets more complex. A mid-sized job might require a small construction crew, with the experience and tools necessary to do the job without alot of standing around while figuring things out. Building things like a bridge or sky-scraper require a large investment in detailed planning, to ensure that the materials and teams of (specialized) workers are in the right place at the right time.

    Programming is no different. As the ambition of the project rises, so does the level of sophistication required to carry it out rise. But the do-it-yourself-ers and small construction crews are not going away simply because we can build bridges and sky-scrapers. There will always be need for projects of every size, and we'll always be best served by using the best people and tools for the job.


    [TINPC@perlcabal.com shh]$ su real
Re: The future of software design
by BrowserUk (Pope) on Oct 21, 2002 at 07:34 UTC

    As others have said, your analogy is flawed.

    I think it is perfectly valid to compare the software industry with either the automobile manufacture or civil engineering. The flaw comes from picking out two specific parts of the two industries for your comparison.

    Software is like cars (think SQL "LIKE" here).

    Attributes Automobiles Software
    General purpose
    Large volume
    "For the masses"
    General Motors Ford
    VW-Audi Renault Fiat
    Nissan Toyota Honda
    Microsoft IBM CA
    Logica
    Fujitsu
    Specialist/luxury Cadillac BMW Lexus Apple - Sun Solaris
    High performance Ferrari Cray
    Enthusiast/homebuild Lotus Seven Linux

    Of course, there could be many more categories and sub-categories, and we could have endless debates about which category any given manufacturer/car/os/package should be in. The point is, despite the marked differences in raw materials/replication costs, when you look at the broad spectrum of the two industries, there are valid analogies to me made. It only when you get into the specific details that these tend to break down.

    <Minor personal rant>

    If the software development and manufacturing industry continues to be able to get away with charging relatively high prices whilst hiding behind "No warranty, stated or implied" licensing agreements and so legally avoid culpability for costs and damages incurred as a result of their failure to test and/or fix, the mess of security and other problems (usually attributed solely to MS, but affecting many other manufacturers too) will continue.

    Can you imagine buying a car if you were forced up front to sign an Agreement waiving responsibility for its safety?

    Without the laws that allowed class action suits to be taken against the auto-industry, we would all still be driving cars with the unenviable safety records of those developed in the 60's and 70's.

    Were there similar legal remedies for failures in software products, it's quite possible that the security scares of Melissa, code-red and many others would have been trapped and dealt with during development.

    Government sponsored safety testing might also help.

    As a footnote in deference to the FSF guys. If you ask your neighbour to fix your cars clutch for a beer, you don't hold him to the same standards as say Firestone whom manufacture and sell mass market products. Nor, if you buy spares for your 70's Mustang via E-bay, do you expect 1 year warranty and full liability.

    </Minor personal rant>

    In terms of the future of the industry, in the same way that many of the mundane tasks of car manufacture--spot welding, crank grinding, panel forming etc-- are now done by robots and other automated machines. I see the software industry moving to automated testing, schema design, log processing etc.

    Logging is a good example of a common software task that is desperately in need of standardisation. The basic requirements of logging are a pretty standard regardless of the application doing the logging. Who, what, when, why, how much. MS os's have (IMO) made a (small) advance over Unix-like os's in this regard with the EventLog mechanism. A standard interface that any application can use to log events in a centrally organised place. This greatly simplifies the detection of inter-application event timing and similar maladies. A utility that allows these to be viewed and queried (in a very limited way) and another API that allows further processing of the logged data.

    Of course, this still stops way short of the ideal. Even with this, its a common task to move the information from the proprietary data store to a centralised (usually relational) datastore so that cross-systems correlation's can be found and reported. Why does every company, big and small, need to home-brew these type of solutions?

    Personally, I think that every OS should have a single, central datastore for "system information". I think I would favour an OO-datastore solution as this seems to have greater flexibility than relational DBMS's. If all system information existed in such a central place, the need for so many scripts that massage one set of data so that it can be utilised in conjunction with a disparate set of related data would go away. Additionally, once a single system's data is stored in a single, standardised place, it becomes a short step to making these system stores talk to each other via the network and cross-systems correlation, control and reporting suddenly becomes much easier.

    (Anyone remember Pick?)

    Note though, that the automating of these type of tasks does not mean the end of software development. It just shifts the emphasis away from the mundane to the interesting. Someone will still have to analyse, specify, code, test and maintain the software that performs these functions. These will, I think, be the desirable (and well paid) jobs in the future of software.

    I also predict a move away from low-level languages like C in favour of higher-level, interpreted languages like Perl (and Java, Python, Rebol, Ruby etc.) I think the next Visicalc (killer application) will likely or not be an interpreted language with a 'compile-to-native-code-and-optimise option', supporting a flexible, extensible, object-oriented datastore. For maximum benefit, this language would have network awareness, data persistence along with code maintenance (pretty printing), version control (CVS), and code production (editor) built in.

    Why, given that Perl knows how to parse Perl, do I need to use 'third-party' products to re-parse the code in order to syntax highlight it, pretty print it or detect the differences between versions? This is considerably easier to do if you do it at the byte-code (op-code) level than at the source level.

    This could, I think, be the future of language development. It could also be the future of Perl 6. Though from my limited perspective, it would require many more hands at the pumps in order to fulfil these 'requirements' in a timeframe that would allow it to be the leader. The amazing thing to me is that the obviously willing and capable resources of 'part-time firemen' that are available to man these pumps is not being exploited.

    Of course, it would also require considerable organisation and management, and with that comes the specter of cost and inevitably charges to offset that cost. I think that this issue could be addressed without the need to compromise the basic tenets upon which Perl was based, given the goodwill of the community.

    However, there may be other compromises that might be less palatable. The need to accept the Real World situation of what systems exist and in what numbers. The possibility of moving away from the GNU licensing arrangement to something that allowed costs recuperation in some manner might be a lot harder for some people to accept. It could also risk dividing the aforementioned goodwill, which would not be a step to take lightly.

    Anyone up for helping me develop an OO-datastore?


    Cor! Like yer ring! ... HALO dammit! ... 'Ave it yer way! Hal-lo, Mister la-de-da. ... Like yer ring!
      On the thread:

      I agree with sentiments stated previously that coding is a 'Craft' as opposed to science/art. It is proven time and again via downloads and code reveiw. TIMTOWTDI, but not all possible solutions are accurate and/or approriate in a given situation for a slew of reasons, yet how many times have you fired up vi, only to be greeted by a cut-n-paste job. Somewhere in this thread was the statement that programming is nowhere near the point where just anybody could pick up a book and code an app. I personally dissagree. No offense intended towards anyone or any language, but Visual Basic anyone? Visual C++? Lets even consider a quasi-code generator for HTML i.e FrontPage. (I'm not attempting a MS bashing here, rather that platform has a predisposition to automate as much as possible for the end user). While the code generated is functional, 9 times out of 10 (or even 99 out of 100) when the 'code' is reviewed by a human, a feeling of utter disgust at the 'code' is felt.

      Personally I wonder where this discussion comes from. There is always a thread on whatever system on the 'net about this exact topic. There have been points where certain groups were 100% positive that "coding" would become a way of the past, while other groups were as sure that coding would never go away. I'm sure that the shift will come, where the focus is higher level writing, but I don't see programming (as it stands right now) ever going away. (At least not until we get massive, parallel, async, neural network driven AI running the day to day business, but then we'll be fighting for our lives not to be 'coppertops' :} ). I just don't understand why this comes up. Yes, programming is a specific, logical process but it's also an avenue for creative expression and giant leaps in terms of efficiency and code size, so where does the need to attempt and pigeon hole it come from? Why? To what end?

      On another note I saw a post about differences in logging facilities, and how windows has a slight advantage over the *nixs due to the Event Logger. Syslog is all I have to say on that subject. But since I'm feeling verbose.. Granted its a little flaky, but its tried and true. Single config file for all system logs, custom definitions can exist for applications without needing multiple conf files all over the filesystem, dependant on what OS is installed. It's not so much the platform's fault as it is people reinventing the wheel on a regular basis. Imagine if all your apps registered a syslog facility, and used syslog.conf for the log file declarations. Syslog is even smart enough to log across the network as well as locally, for centralized log storage/processing. Instead people decide to do it their way, or don't trust the syslog facility. Maybe it's due to security? Maybe someone could whip out a SSsyslog, making use of application keys or some such to secure/encrypt certain logs or network transmissions. I dunno, just thinking out loud. Again this isn't so much a rant as an attempt to edjucate.

      On to the slightly off topic portion...

      I would *_LOVE_* to work on an OOP implementation for a centralized app to keep track of a system's (meta)? data about installed components/applications. It's been a pet peev for a long time. Please contact me (kashimiku@hotmail.com), or message me, or start a thread, or whatever else it takes to get this rolling.

      /* And the Creator, against his better judgement, wrote man.c */

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://206214]
Approved by Zaxo
Front-paged by tadman
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others having an uproarious good time at the Monastery: (11)
As of 2014-04-23 22:56 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (556 votes), past polls