Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Re: OT: When to put things live

by Old_Gray_Bear (Bishop)
on Apr 19, 2007 at 17:26 UTC ( #611003=note: print w/ replies, xml ) Need Help??


in reply to OT: When to put things live

That's pretty much the size of it. We aim for 'non-disruptive enhancement' around here. That means:

  • No Friday installs of anything that has a Customer facing component.

    Of late this has become problematic, since we have a sizable User base distributed around the Globe. Someone is always inconvenienced, no matter when we install; so we have settled on Developer Convenience as the determining factor. This means that the last five or six Product releases have occurred on Wednesday Evening between 1800 and 2000.

  • The Service DOES NOT GO DOWN.

    The New Service (Client enhancement, bug fix and code maintenance, new database features, etc) will run in parallel with the Old for a period of weeks. (Or a period of years, in the case of Client code. We have Client-software in the Wild that is over eight years old. The feature set is still supported, We don't accept bugs on it, however.)

  • While there is a committee involved in the planning and scheduling of a Change, there is a single Change Captain. The Change Captain has final authority to say 'Oops. Back it out'.

  • We try to give our User Community a reasonable estimate of when a new feature set will be available. We plan to have it in play four to six hours before the announced go-live. This gives us a little 'final-checkout' time. Our Users know this and so we sometime get 'early adopters'. We don't discourage this.

  • The Usual Time Line:
    1. Two weeks before the date -- feature freeze
    2. One week before the date -- code freeze and QA begins regression testing.
    3. On the day:
      1. 1300 -- Final Change Review meeting -- are we really ready?
      2. 1600 -- New code/hardware active and checked out
      3. 1800 -- Go-time -- everybody involved gathers in a conference room and watches the logs and monitors.
      4. 1810 -- Pizza delivered (on the Project Managers nickel)
      5. 1930 -- More Pizza, this time with beer (ditto; there is a line item in the project budget for this)
      6. 2000 -- Go-Live for the Users; Ice cream arrives
      7. Afternoon of the following day -- Post Mortem
This seems to be a working method, it has served for the past fourteen months. We have only had one release aborted by the Change Captain -- when it was announced in the 1300 'final readiness' meeting that the primary power system to one of the co-location facilities had failed at 0300, and we were on standby generators. The power vendor 'expected to have it back online by close-of-business today.' The CC said "that's nice. We ain't going until the generator has been up for at least 12 hours." We slipped the install a day.

----
I Go Back to Sleep, Now.

OGB


Comment on Re: OT: When to put things live

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://611003]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (3)
As of 2015-07-07 02:29 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (86 votes), past polls