I hope that I am not abusing this forum. But I feel that people that make this community have much more than just Perl expertise. My question is, "What are requirement documents?" I think I know that there are typically: "Requirement Document", "Detailed Design Document" and that's about it right? What else it there? What is "traceability and a traceability matrix"? What document do the test plans belong? Who should write these documents? On the monster job site they list this as on of the skills of a software engineer.

The ability to create functional and technical design specifications for development efforts is an essential skill for software engineers.

What are functional and technical design specifications?

Does any know of any good books or web-sites that define these documents? I need this information to deal with a every increasing contentious atmosphere here at my work. As the deadlines go racing by and my "team" has failed to produce a deliverable product, I wonder if this is due to attitude of the project leadership that "documentation" is a luxury that we didn't have time for.

Replies are listed 'Best First'.
Re: requirement documents?
by derby (Abbot) on May 12, 2003 at 17:17 UTC
    One of the few decent things out of MS is MS Press. Check out Software Project Survival Guide.

    To answer some of your questiosn:

    • requirements doc - a description of what the software should do (sometimes called the functional spec)
    • detailed design doc - a description of the low level implementation (sometimes called the technical spec)
    • traceability matrix - a document that pins test scenarios to items in the requirements doc (ensure what's built is what's asked for)

    Other docs - depends on the model being followed. But you could have

    • Software Development Plans
    • Schedules and estimates
    • Interface Guidelines
    • Qualtiy Assurance/Testing Plans
    • Deployment Plans
    • Architecture Plan
    • User Manuals
    • Change and Control Plan
    And a whole lot more.

    -derby

      I'm reading and enjoying ISBN by the same author as the above. It has good information on the various preparation stages of software construction that seem interesting.

      OT: Hmm..according to preview The Amazon tag might not exist yet. :) I checked the conveniently linked "Writeup Formatting Tips", but I couldn't easily find a link to a page about "bracket" tags I can use from there.

      Mark

Re: requirement documents?
by vladb (Vicar) on May 12, 2003 at 19:23 UTC
    Good that you asked... I'm now working on both the specifications and design documentation for a medium-sized project. This project involves development in both Java (for the front-end Tomcat/Velocity VCC framework) and Perl (for the back-end feed parser / manager). The perl component was there before and one objective of this project is to re-design the whole piece and hopefully make it a lot more simpler.

    I started out by building a functional flow-chart for the original Perl feed parser process. On it, I tried to concetrate on particular areas of interest. One example is where/how parsed data is being stored into the database. Once ready, I submitted it for a review by my colleagues and immediate boss. As soon as I had received a favourable responce, I proceeded with my work on a revised copy of the flow chart. This one had to reflect the inner-workings of a simplified/improved version of the feed parser process. In their final shape and form, the two flow charts and accompanying specification are expected to provide a clear picture of the system and any work that might be required to improve it as per project requirements.

    Following my work on the flow-charts, I then was able to identify specific tasks that would take the project from start to finish. The detailed design doc was key in determining those tasks. Having to come up with an hourly project time schedule is a hard task all in itself. However, having to do this without a thorough and detailed design document is virtually impossible.

    update: Added a *missing* sentence (2nd paragraph) and fixed a few shameful grammatical errors :)

    _____________________
    "We've all heard that a million monkeys banging on a million typewriters will eventually reproduce
    the entire works of Shakespeare. Now, thanks to the Internet, we know this is not true."

    Robert Wilensky, University of California

Re: requirement documents?
by benn (Vicar) on May 12, 2003 at 20:00 UTC
    There are as many "documentation schemes" as there are companies - everybody wants it done "the way *they* learnt" or "the way it's done in XYZ", and we often tend to end up with a mixture of documents with crossover functionality and often contradictory information.

    I recently completed a job for a large UK PFI company, who attempt to follow civil service documentation standards, (including couriering 'sensitive' design docs because they hadn't learn about encrypted mail...) along with a few wrinkles of their own. It was a nightmare. 5 different documentation departments, each with their own set of standards. Requirements passed to Functionality who passed to Low-level Design who passed to Interface Design...

    It left me thinking that most projects need just two documents - a specification, and a user manual - "What we want" and "How we did it". (At a pinch, just the specification, which in theory could serve as the manual too, once translated into User {g}). A requirements document can be quite happily signed off, then sent to the design department. They can turn this into a completely unrealistic set of achievables, simply from their choice of implementation menthod - one, which furthermore, can completely contradict the document coming in from the hardware guys. Your implementation could follow both the requirements spec and the hardware spec, but fail the design requirements. You know the old cartoon with the rope swing?...

    If however, you're lucky enough to get all your people round a table at the same time, and hack out a single document, then you've got your Bible. This would include a test plan, and an implementation everbody was happy with - I've seen a project take an extra month simply because the database design doc. wasn't part of the initial process, and 3 months in, they decided to impose their own standards.Timescales can be thrashed out, you can follow it to the letter, and get paid accordingly. With a comprehensive single signed-off document, you know when the job's a good 'un, and you're finished. All the boxes in the spec are ticked.

    Good luck with the deadlines,
    Ben.

      I would like to comment on just this one sentence: "If however, you're lucky enough to get all your people round a table at the same time, and hack out a single document, then you've got your Bible." Do you really need physical presence of all the parties? Would not a web collaborative tool (like a simple Wiki page) be sufficient? Does anybody have experience with using those tools this way?
        Do you really need physical presence of all the parties?

        In my head, it was metaphorical originally, but it's an interesting point. I work/meet online, have been doing so for years, and am completely comfortable with collaborative online tools. Many 'managers' though are not, and insist on real meetings in a real office (obviously, it's a good way for them to look like they're working :) ).

        I think 'real' conversation - including coffee / cigarette breaks, OT chats about family etc. - can engender a sense of teamwork that is difficult to achieve when simply inputting text into a Wiki. As to whether this improves efficiency, I'm not sure, but I should think it'll be 10 years yet before online collaboration is the norm - basically when Monks start wearing suits and managing :)

        Cheers
        Ben.

large team v's small team development
by g00n (Hermit) on May 13, 2003 at 01:22 UTC
    Does any know of any good ... web-sites that define these documents

    A good site to see *real* as opposed to theoretical software engineering is www.joelonsoftware.com. There is enough good information for you to put aside the fact that Joel is an ex-microsofty ;)

    What are functional and technical design specifications?

    you can read more about specifications here (1. why, 2. what, 3. how, 4. tips) but this is where it gets a bit murky. Specifications mandated by traditional software engineering work well for large teams working on *certain types* of projects - (Joel was product manager for MS Excel embedded language specification - VBA) But these techniques are not as efficeint for small teams whose deliverables may not be shrink-wrapped.

    what about anti-specifications?

    Extreme programming on the other hand is a light weight development process that avoids some of the pitfalls of *documented specifications* especially where you are developing in small teams (1 >= member <= 30) where rapidly changing *business, technical and functional* requirements are the norm for a non shrink wrap market.

    So ask yourself what type of development will I be working with (large, small). What type of product will I be releasing (throw-away, internal, embedded, shrink wrapped?). Then check out the various different approaches to specification requirements in the links above.

Re: requirement documents?
by BrowserUk (Pope) on May 13, 2003 at 08:47 UTC

    Most of the question about what and why have been answered, but the outstanding question is who.

    Every software development project should have a writer. For a big, well funded project this person may have the title of 'technical author'. For smaller projects, including 2/3/4 person efforts, a school leaver (*) or a part timer is ideal.

    * I've been reliable informed that the term "new or fresh graduate" might be less humourous in US parlance.

    They only have to have three qualities

    • The ability to use a simple editor or word-processor at a reasonable typing speed.

      A reasonable standard of written english (or other project language).

    • A willingness to learn, a thick hide and persistence. A 'sunny disposition' can help provided they are not too bouncy on Monday mornings before say ... 5.00 pm:)

    And one non-attribute: They mustn't be a coder of any form

    They must then be invested with sufficient authority (preferably from within the group, not above it) to be able to nag any and all individuals for information.

    Their job is to write down, and type up whatever information, documentation etc. is agreed as being required, and getting it done to the schedule that is agreed up front.

    And that means that they have the authority to sit beside the developer and ask questions whilst the compiler is compiling, the coffee is brewing, the Dilbert page loads, and whilst the developer is studying the insides of his eyelids in the hope of inspiration.

    Anyone who says that their project cannot afford such a person is wrong! Trade the time lost in having developers document -- ie. write macros for the word processor, spending hours getting the diagrams exactly right, and going into excrusiating detail of how the hidden scrolling atttributes pop-up works -- against the wages of a non-coder, and the math always favors having such a person.

    The benefits of having just one person who can tell you the state of play of each of the agreed documents, what (who!) the hold ups are, and chasing up approvals, if applicable, are incalculable.

    Coder hate to document, and when they are forced to, they will spend inordinate amounts of time 'fixing the broken WP/editor', producing better graphs, tables, diagrams, layouts, macros and whatever OR write stuff that is entirely unreadable by anyone without a CS/coding background and intimate knowledge of X, Y & Z.

    The writer, not being of this background, will need stuff explained in simply, clear terms and should be encouraged to write it down that way, in their own terms -- just right for Managers and Users alike:)


    Examine what is said, not who speaks.
    "Efficiency is intelligent laziness." -David Dunham
    "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller

      I think you missed a close font tag. :-) Excellent post. I have been saying this (to no avail) in my present team/company for a long time. All those people who think making developers document their own code is a good plan are nuts. My experience is that developers usually have several orders greater understanding than the user/management does of the systems they are working on and computers in general (no suprise here). What is a trivial matter not worth even mentioning to a developer can be a point of crucial misunderstanding leading to a show stopper for a user. To expect a developer to even be able to lower their mind to the level where they cover what needs to be covered with the detail appropriate for an end user is a great way to ensure your documents are unusable. Sure there are developers out there who can do this, but they aren't common.

      This remind me of a story from my long distant past. I was doing some remedial mathematics and was having some problems with some stuff. So I went over to a see a good friend, who was at that time in the first or second year of his math PHD. He _tried_ to help me with my homework, but in the end we gave up. The reason was that he just couldnt forget his skills. One question was expected to result in around a page of sums, he stepped in and used some rule they dont teach you untile 2nd or 3rd year Math that reduced it to something like 2 lines. Of course this was no good to me. My point here is that expecting a developer to write documentation that is usable by a noncomputer expert is like asking my friend to forget all the math he had ever learned and to return to thinking along the same lines he did in high school. No chance.


      ---
      demerphq

      <Elian> And I do take a kind of perverse pleasure in having an OO assembly language...
      Every software development project should have a writer. For a big, well funded project this person may have the title of 'technical author' ... Coder hate to document ... write stuff that is entirely unreadable by anyone without a CS/coding background and intimate knowledge of X, Y & Z.

      There is some truth to the comments here. But I will add, "who are the documents for?".

      Coders need specifications where everything spelled out in great detail. Be it formal specifications or an agile process you have to be able to express the result in binary. There is no room for interpretation. If you are on a large team and assigned Foo class you may well be interested where it fits in the class heirarchy - something a BA or non-cs/it writer cannot reasonably provide.

      Management types want to know *what they are spending their money on* and *is this product alligned with my business objectives*. It is here that a *writer* may be of some use, distilling the *bar* product into some language that makes business sense.

      Users dont read documents and only want documentation if *all else fails*. But end user documentation is worth it's weight in gold. Many a project has failed because users cant just *read the source*. Hence writers again prove useful in generating documentation.

      As a result I never really mandate any one *documentation/specification* process or set of *hard rules*. Each project is different. Instead I try to provide the right balance of documentation between coder, phb and user.

        Okay. We agree on the case for documents destined for Management types and Users:)

        You have a problem with the idea for (for want of a better term) 'technical documents'.

        I would still contend that having a 'third party' do the writing is not just beneficial, but essential. Though I do agree that the writer in this case may well need some technical expertise. The two best technical writers I have worked with both abhorred anything to do with programming, and the worst was a frustrated programmer!

        The problem with having coders write their own documentation, is that they approach the problem from a coders perspective. The result is, that you end up with documents that need a folding editor, replete with hypertext linking and a recursive descent "de-geeking" parser to read them. This is why so much technical documentation is so darn difficult to read and use.

        The skills, knowledge and mindset required for producing complete, concise, accurate distillations of large volumes of technical information, especially if this will ultimately be viewed in a flattened form (ie. paper), are considerably different to those involved in writing programs, and this should be recognised. A good technical writer does not have to understand the systems, processes, techniques or datastructures that they are documenting. Their skill is in extracting the salient information (with pliers if necessary:) and organising it into a logical, maintainable format. And being dedicated to that task regardless of the priorites and emergencies in the development process.

        Spelling it correctly, indexing it well, and making it look nice in terms of layout are nice-to-haves, but this is a secondary function that can also be done by a WP/DTP specialist who has absolutely no understanding of the content.

        My reasoning for removing the burden of documentation from the programmer is not that it is secondary to the code. Far from it. It is at least as important as the code. In the longer term especially. What my reasoning does say is that as it is so important, it should not be left to the coders to do because they will always consider it secondary. Recognising that producing documentation is a seperate, highly skilled and valuable part of the development process means that it should be recognised as a seperate function.

        The reasons for suggesting that (in small shops with tight budgets) a recent graduate might ideal for the post are

        • They need to gain experience and command lower incomes as a result (they are cheaper).
        • The very skills that the recent graduate has (or at least should have:), self organisation, the ability to perform research, sticking to a timetable etc. are the very skills most needed in organising documentation.

        The bonus (for them), is that they get an inside view of the development process and produce documentary evidence of their involvment. In most cases there are at least one or two of the documents produced that the graduate can be allowed to take away, as examples of their work, without risk. These can serve a prospective new employer as a strong indicator of their mindset and abilities way beyond their use of word processor.

        In just the same way, a medium-sized sample of a coders code can serve as a good indicator of their skill level, regardless of whether it is of the same type of code, or even in the same language as they would be required to write in their new position.


        Examine what is said, not who speaks.
        "Efficiency is intelligent laziness." -David Dunham
        "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller
Re: requirement documents?
by talexb (Canon) on May 13, 2003 at 15:48 UTC
      What are functional and technical design specifications?

    A functional specification explains What it should do and the design specification explains How it's going to be done.

    --t. alex
    Life is short: get busy!
Re: requirement documents?
by MadraghRua (Vicar) on May 13, 2003 at 18:17 UTC
    HiYa
    I and my group use the Rational documentation process and follow their templates. Look here for an idea of what they do. Bookwise check out the Addison Wesley Object Technology Series with Booch, Jacobsen and Runbaugh as the series editors. A good book to start with is 'The Rational Unified Process An Introduction, Second Edition' - this is required reading within my group. 'Managing Software Requirements' by Leffingwell and Widrig of the same series is a good practical book on requirements gathering and documentation, IMO.I suspect if you look around the web for a bit you'll see templates to use somehwere.

    MadraghRua
    yet another biologist hacking perl....