Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw

Re: OT: When to put things live

by Old_Gray_Bear (Bishop)
on Apr 19, 2007 at 17:26 UTC ( #611003=note: print w/replies, xml ) Need Help??

in reply to OT: When to put things live

That's pretty much the size of it. We aim for 'non-disruptive enhancement' around here. That means:
  • No Friday installs of anything that has a Customer facing component.

    Of late this has become problematic, since we have a sizable User base distributed around the Globe. Someone is always inconvenienced, no matter when we install; so we have settled on Developer Convenience as the determining factor. This means that the last five or six Product releases have occurred on Wednesday Evening between 1800 and 2000.

  • The Service DOES NOT GO DOWN.

    The New Service (Client enhancement, bug fix and code maintenance, new database features, etc) will run in parallel with the Old for a period of weeks. (Or a period of years, in the case of Client code. We have Client-software in the Wild that is over eight years old. The feature set is still supported, We don't accept bugs on it, however.)

  • While there is a committee involved in the planning and scheduling of a Change, there is a single Change Captain. The Change Captain has final authority to say 'Oops. Back it out'.

  • We try to give our User Community a reasonable estimate of when a new feature set will be available. We plan to have it in play four to six hours before the announced go-live. This gives us a little 'final-checkout' time. Our Users know this and so we sometime get 'early adopters'. We don't discourage this.

  • The Usual Time Line:
    1. Two weeks before the date -- feature freeze
    2. One week before the date -- code freeze and QA begins regression testing.
    3. On the day:
      1. 1300 -- Final Change Review meeting -- are we really ready?
      2. 1600 -- New code/hardware active and checked out
      3. 1800 -- Go-time -- everybody involved gathers in a conference room and watches the logs and monitors.
      4. 1810 -- Pizza delivered (on the Project Managers nickel)
      5. 1930 -- More Pizza, this time with beer (ditto; there is a line item in the project budget for this)
      6. 2000 -- Go-Live for the Users; Ice cream arrives
      7. Afternoon of the following day -- Post Mortem
This seems to be a working method, it has served for the past fourteen months. We have only had one release aborted by the Change Captain -- when it was announced in the 1300 'final readiness' meeting that the primary power system to one of the co-location facilities had failed at 0300, and we were on standby generators. The power vendor 'expected to have it back online by close-of-business today.' The CC said "that's nice. We ain't going until the generator has been up for at least 12 hours." We slipped the install a day.

I Go Back to Sleep, Now.


Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://611003]
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others studying the Monastery: (4)
As of 2023-11-30 17:25 GMT
Find Nodes?
    Voting Booth?

    No recent polls found