Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Migrating Code From Test to Production: How to do it right, how to set up an environment that leaves nothing to chance

by tphyahoo (Vicar)
on Jul 11, 2006 at 10:06 UTC ( #560384=perlmeditation: print w/ replies, xml ) Need Help??

My meditation for today is a link to a brief, but in my opinion classic, post from dragonchild that I think could use some more attention.

Re: What is YOUR Development Process?

When I posted it to reddit with this title (good titles are everything there), it got quite a bit of approval. And it got delicioused by 81 people within a couple of days. So, I figured it was worth reposting it to perlmonks, where I originally stumbled across it.

Thanks, Dragonchild. :)

Comment on Migrating Code From Test to Production: How to do it right, how to set up an environment that leaves nothing to chance
Re: Migrating Code From Test to Production: How to do it right, how to set up an environment that leaves nothing to chance
by Herkum (Parson) on Jul 11, 2006 at 13:30 UTC

    I wrote a bunch of tests that depended on a database for holding my data. The tests deleted all the rows from the tables that I was working with and then inserting the test data into the tables. If I did this against the production database I was going to delete the production data as well, that would be bad.

    I decided to have a production and a test database. I wanted something that would be easy to include in my test code and would not require a major change in production code. I thought, if I am working with my test modules I make my DB connection code look like this what would I have to do.

    use Base::DB::Connect 'test';

    The only thing I had to do was include a import subroutine/method. It changed the $database variable from 'Prod' to 'Test' and I was done! I was happy because I did not have to change the code to support 'Test' and 'Production' as separate environments.

      What is an "import" subroutine? Why did you boldface this term? Does it have a special meaning?

      What terms do I search the web on to learn more?

        It has to do with the use function, which is really just syntactic sugar. One of the things it does is to call the static import method on the package youíre loading, passing it the parameters you gave on the use line to process them. The common behaviour of import is to export functions from the package itís in into your current one, usually by calling upon Exporter. However, all of that is entirely conventional, and you can actually do whatever you want with those parameters Ė like using them as a names for configurations to choose between, as Herkum does.

        The use docs have the details.

        Makeshifts last the longest.

Re: Migrating Code From Test to Production: How to do it right, how to set up an environment that leaves nothing to chance
by ptum (Priest) on Jul 11, 2006 at 15:22 UTC

    I have worked for several companies which have tried to establish a 'representative' development environment, only to find that there were significant differences between the development and production environments which made code migration problematic.

    When it comes down to it, I would argue that there is no substitute for actually trying your software in production. For that reason, I try to write my code in a partitioned manner so that I can migrate nearly all of my code to a production environment without actually going 'live'. This may require a database switch that causes the 'write' portions of your code to point to a test database, or even (as may be the case with some major online retailers) require you to partition off a redundant portion of your production environment for a 'one box test'.

    I used to go to great lengths to package up my software in a Solaris environment so that my code could be 'cleanly' installed by operations folks while I stood nervously in the background. I spent a lot of time writing the installation scripts, and I never did quite account for every possible environmental difference. Whenever I can, I leave nothing to chance by being the guy who migrates the software and having a full set of test scripts that exercise the code once it is installed in production.


    No good deed goes unpunished. -- (attributed to) Oscar Wilde
      When it comes down to it, I would argue that there is no substitute for actually trying your software in production. For that reason, I try to write my code in a partitioned manner so that I can migrate nearly all of my code to a production environment without actually going 'live'. This may require a database switch that causes the 'write' portions of your code to point to a test database, or even (as may be the case with some major online retailers) require you to partition off a redundant portion of your production environment for a 'one box test'.
      I really like this design, and I employ it myself. I find it very, very handy to be able to "go live" after I have tested the system in production (but not generally seen by everyone).

      An "on" switch has other benefits: I value the ability to "launch" at a particular time that meets the customers expectation. This would be otherwise difficult to do because installations take time. "On" switches are virtually instantaneous

      Conversely, this implies an "off" switch which can be useful in some circumstances.

      -------------------------------------
      Nothing is too wonderful to be true
      -- Michael Faraday

Re: Migrating Code From Test to Production: How to do it right, how to set up an environment that leaves nothing to chance
by ruoso (Curate) on Jul 12, 2006 at 10:55 UTC

    One thing I learned is to use chroots for test environments (This only applies for unix-like systems, I think). Ideally, you'll have a chroot for each version that you should care about (production, integration test, development test). Each chroot has a copy of the webserver *and* the database, and before an integration test, make a copy of the chroot so you can repeat the test. This is saving me a lot of time, as, to fix a bug in production, I can just spawn (creating a copy first) my production chroot (which is a clone of the real production) to find and fix the bug.

    This is combined with having a Tag in the revision control system (in my case, CVS (Yes, I know many are switching from CVS to SVN, but I do like the file-oriented nature of CVS)) for every version you care about. In addition to what is said in the link the OP cites, I consider the HEAD as "sacrosant" also. Ideally, HEAD would not only "at least compile", HEAD *MUST BE RELEASEBLE AT ANY TIME*, to do that, every development is made on a branch tag and is only merged to HEAD when all tests passes.

    Ah, the fact of having a tag for each version, makes it possible to take the production version to only fix the bug in question without "contaminating" it with the new features^Wbugs...

    daniel
Re: Migrating Code From Test to Production: How to do it right, how to set up an environment that leaves nothing to chance
by samizdat (Vicar) on Jul 12, 2006 at 13:25 UTC
    Ahhh, the crux of the matter! Indeed, good subject to revisit, tpyahoo.

    I always integrate my customers with my deployment process, especially when I'm working with a live database. There's rarely a situation where transactions can't be queued for a few minutes, at least at three AM: long enough to do a block -- data dump -- install -- test -- release block. You do have a queue and queue handler on your database, do you not? :)

    Warnings are always posted and propagated a day ahead of time, and our web systems post alerts on all database-active pages.

    Finally, our systems involve replication and hot spares, so we're used to having more than one live system. We introduce a newly upgraded machine into the mix, point dynamic DNS at it, and watch the transaction logs. If it fails with data coming in, we simply pull its transaction logs and feed them to one of the existing servers after taking it back off-line.

    I work as hard on the 'revert' scripts as I do on the 'install' scripts. You'll never catch everything, so be ready.

    Don Wilde
    "There's more than one level to any answer."

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://560384]
Approved by Arunbear
Front-paged by Old_Gray_Bear
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (7)
As of 2014-10-26 04:54 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    For retirement, I am banking on:










    Results (151 votes), past polls