After reading Ovid's piece on specifications, I thought it might be appropriate to follow up with a rant/horror story on specifications, testing (Beta, and other) and quality assurance. I will begin with a true (and currently ongoing) situation (textual color added for the sanity of the writer).
- In which our hero gets sucked in
Allow me to take you back two months, nay three, to a day where we find our intrepid adventurer (that's me) sucked into a new and exciting application (that is already in stage two WITHOUT written specifications..Ovid, I feel your pain). Weary and worn from years of battle, our illustrious hero stalks hesitantly into a council meeting with those of the elite white collar class. Lacking any knowledge of the arcane or warrior arts, yet wielding staggering decision making abilities, these white collars were on a mission to wreak havoc on our hero's production schedule in the guise of "streaminlining" a current business process.
Reluctant to continue without a specification, but being confronted by those who "needed no specification to continue", our hero plods through the meeting, all the while wondering as to the origin of his next paycheck. The project, it would seem, will deal with a Word document that is currently being emailed around the globe each morning to the inboxes of executives and their assistants. It contains news articles regarding the empire and it's interests, and is apparently important to all involved. This document, however, although important and well read, is large and taking up exchange server space, and must be dealt with. The imperial white collars wish a process to be put in place where the lackey who creates this document can merely upload the document to the web, with the result being an email with a link to the contents of said document.
Our hero suggests a method whereby a server application could pull the document apart after upload, load the seperate articles into a database, and offer an interface via the web through an .asp or /cgi/ application and email a link to said interface. The white collars are both intrigued and, well, confused by the level of technical and arcane vocabulary whirling through the room, and ask..."Yes, but, will it do what we asked?"
Noticing, not for the first time, the confused looks, knit brows and blank stares he sighs and nods, "Yes," he says. "Yes it will, but can I have your request in writing for my files?" By then, however, it was too late, as he has been dismissed with a subtle nod and a changing of the subject to the Executives' respective children, power boats, vacation plans, and other important business topics. Our hero leaves the council meeting regretting leaving his cube, and heads back to it's relative safety, ready to harness the arcane and begin the code, yet again without a specification document, written request, or whiskey stained bar-nap with bleeding doodles as a guide.
- In which our hero codes, loads and prepares
A month has passed, and our hero has been hard at work in the deep, dark, depths of the cube warren creating exactly what was asked for...or so he thought.
As the development phase of the application neared an end our hero approched an underling of the white collars, more specifically the person whose idea this arcane cluge was, and asked that she compile a list of test subjects on which the code could be tested. Preferrably a list of those whose screams of distaste, fear, loathing and ululating joy would be loud, profound and DETAILED. She returned with a list of names ranging in power and stature from the emporer's advisor to the emporer himself. Our hero blanched, and asked that she humbly rethink her choices, as the list she had chosen would prove to be one of closed mouthed and distracted users, untested in the art of beta, and probably unwilling to try something new. She declined, and sent an email to everyone on this new document distribution list, stating that a new process would be put in place over the next month in tandem with th ecurrent process, and could the users please provide feedback.
By this time, the application has gone onto a live server, and is ready to beta test. Our hero has created a web interface using .asp per request, and implemented a fairly robust yet specialized parser for Word documents in perl and Win32::OLE. The parser receives a file, parses it based on certain formatting guidelines specified by the document author, inserts the articles into a database and emails a link to the table of contents .asp listing the articles in the document of said date. The front end as mentioned, is accessed through a series of .asp pages that retrieve articles, table of contents, etc, based on date. Everytime afore mentioned underling uploads a days document, a new email is automatically gerenated and sent to the list...or so our hero thought.
- In which our hero finds that to make God laugh, tell God what you are planning for tomorrow
Our hero created a tidy little text file with all of the names for the distribution list on the server so that it could be easily maintained by the underling until a distribution list could be created on the exchange server (something that is technically out of our hero 's control). This file was gerenated from a list of names given to him by the lackey.
Technical Note #1: Exchange saves a users email prefix in a field called ALIAS, in the following form: <CCharbeneau> Please take careful note of the CASE of the letters. That's first letter of first name, and last name. In my case Chuck Charbeneau. HOWEVER, it will not accept it in the form of CCharbeneau@lear.com from a remote connection...You have to force lowercase...This I did not find out, to my chagrin, until later.
Early on in the beta process, our hero received an email from the lackey's boss (the original requestor of this little gem of an application) saying that she did not get an email. As she had a difficult and easily misspelled name, and other people WERE receiving the email (but not EVERYONE, we would find later...), it was copied and pasted from an email into the list again, but still without results. Mumbling something regarding "Line feeds and the like..." Our hero re-types her name into the file by hand, this time producing the desired results. In response to her email, our hero asked if anyone else was likewise not receiving the email, and received no response.
A few more weeks passed, and another email is received from the 'Requestor', asking if our hero has received any feedback. He replies to the negative, referencing their earlier conversation regarding the usefulness of the chosen test subjects, to whit she responded, "No news is good news". This was, in the end, bad news for our hero.
A few weeks passed, and no feed back is received, so the CIO, our hero's master's master's master's master, enters into a meeting of the minds with a number of the members on that beta test list, mentions the application, and finds that none of them had ever received an email with a link in it. No mention of the application what-so-ever since the initial email suggesting that a beta was going to happen. Needless to say, a panic ensued. Buttons were pushed, phone calls were made, lights and alarms sounded, and our hero was yanked before a group of "concerned" white collars. With a small amount of investigation (and a few "I told you so's to those in power")our hero discoverd the root of the problem (cf. Technical Note #1) and made the necessary code changes, and all was well...or so he thought.
The White Collars decide that instead of a beta test, that the project will now just have to go live, and our hero is told to "just make sure there aren't any more screw ups"...
PROBABLY NOT THE END...
Isn't that what beta testing is for??
The moral of the story, you ask?? Always get specifications...Always communicate with the project requestors (PR), and always, always DEMAND feedback from a beta test group, even if you have to force someone's lackey to do it. That was my BIGGEST fault in all this. I trusted the PR to get feedback. However uncomfortable it may seem, I should have had some interface with the user base, whether or not they could blink me into non-existance.
The story actually has a couple more twists and turns, including a feature request 2 hours into my mad dash to fix the email SNAFU, and a question from the Chairman of the Board as to why the process had changed, he liked it better the old way, and could he still get the Word Doc emailed to him in the morning....
update: For part II see this