See http://petdance.com/perl/ for a couple of my talks on the topic of large-scale project testing, including the one I gave at last year's OSCON.
in reply to Testaholics Anonymous (or how I learned to stop worrying and love Test::More)
Also, I'd be glad to visit your local Perl Mongers group and talk, too. (No cost to you, either, so long as my costs to get there don't exceed $100) I'm going to Minneapolis in a few weeks, and I've spoken at St. Louis and Grand Rapids.
Short version of my comments on this thread:
Here's the bot we use at work. It's set up to handle a branches if necessary, as in:
- Redundancy is OK. Don't build for redundancy, but if you happen to have redundant tests between a couple of .t files, that's OK. Don't try to remove the "inefficiencies".
- Don't worry how long your tests take. Set up a bot that runs hourly and notifies you (or the entire department, in my case) if any tests fail.
Adapt to your own uses:
# Runs the smokebot program on the trunk and mails the results
0 * * * * smokebot HEAD email@example.com
# Smoke against the branches
30 * * * * smokebot cp2004-branch firstname.lastname@example.org
if [ $# -lt 2 ]
echo Must pass at least a branch, and one email address,
echo plus any parms to echo pass to smoke.
# This assumes you have already logged in once as anoncvs
# so that the password is in your ~/.cvspass file.
cvs -d/home/cvs -Q co -d $DIR -r $REV tw > /dev/null
/home/smoke/tw/Dev/devapache stop > /dev/null 2>&1
/home/smoke/tw/Dev/devapache start > /home/smoke/smoke.out 2>&1
smoke $@ >> /home/smoke/smoke.out 2>&1
grep -i "^Failed" /home/smoke/smoke.out > /home/smoke/smoke.out.fail
if [ -s /home/smoke/smoke.out.fail ]
mail -s"Smoke $REV $@ $STATUS `date`" $MAIL < /home/smoke/smoke.ou
/home/smoke/tw/Dev/devapache stop >> /home/smoke/smoke.out 2>&1