Perl Testing and Quality Assurance

[Anonymous Monk]

I'm getting back to Perl programming after a decent chunk of time away from it. Looking back at my old code is making me exceptionally nervous. I haven't noticed any bugs in the last 4 months of running some of it, but I'd like to invest a little time in increasing my peace of mind so I'm looking at different testing approaches.

Looking at some of CPAN's results shows there's no shortage of module options (as I've come to expect from Perl). My question is what recommendations do the monks have for someone new to testing? Which modules should I check out? Thanks.

[adrianh]

My advice would be to use the Test::Builder based options.

Start with Test::Simple and Test::More. Test::Tutorial provides an introduction to these.

After that take a look at some of the more specific testing modules like Test::Data, Test::HTML::Content, Test::Pod, Test::Warn, Test::Exception, etc.

If you want a more xUnit type of testing framework take a look at Test::Class (note: I'm biased since I wrote that one).

All of the above, and many more, are build on top of Test::Builder and all play together well. Kudos to schwern and chromatic for a great library.

In the perlmonk land you may find the following of interest:

• The Joy of Test
• Automated software testing: emulation of interfaces using Test::MockObject
• Unit Testing Generated HTML
• What goes in a test suite?
• Further adventures with testing
• Micro Mocking: using local to help test subs
• Small test harness for testing basic auth

As a bit of more general advice try switching to doing Test Driven Development. It feels odd at first but does wonders for code quality once you get used to it (IMHO anyway :-)

[Zaxo]

More difficult than choosing your testing toolset, you will need to decide what your tests should do. It's too late for coding to a specified test set ( which I completely agree with adrianh about - it's like magic ). Your code already exists and is in use.

I would begin by reviewing and rewriting the documentation. That will set in mind what the user interface is supposed to be like, what the internal API is to do, and will clarify the design in your memory.

Write user-level tests. In those, verify that the high-level documented interface works as advertised. Test erroneous usage for sanity. Then set these tests aside without fixing errors or improving anything.

Now start devising unit test of each sub. Your documentation of internals should guide you in what each should do, what error conditions may arise, and what tolerance each shold have for those errors. Do tests of each, from the bottom up. Induce error conditions to check for correct handling. Don't forget to test global data, too.

If your documents don't tell you what to test about a feature, that's a sign of probable hot spot. Study that closely and find out what the details should be.

Don't be too eager to change your design, but do keep an eye open for when you must.

After Compline,
Zaxo

[adrianh]

All excellent advice.

Another approach to determining what your tests should do is to see what you can remove from the code.

Just delete a line or subroutine.

Do all your tests still pass? If so one of the following is true.

• You need to write another test to highlight the defect in the code you've just created.
• You've found a feature that's surplus to requirements - well done you now have a simpler code base.

God I love unit tests :-)

You can even automate this sort of thing if you are that way inclined (see this interview where Java's Jester is discussed for an example).

[dws]

My question is what recommendations do the monks have for someone new to testing?

If you're new to Unit Testing, I recommend reading one of the "walkthrough" books. You'll come away with a some new tricks in your bag, even if the target language isn't Perl.

Kent Beck's Test Driven Design: By Example is good, as is Newkirk and Martin's Extreme Programming in Practice. (I thought this was a one-read book, but I'm now going through it a second time.) Also worthwhile is a series Ron Jeffries is working up on Test Driven Development in C#.

Unlike many books that show only the finished product, these works show false starts and misteps, and how the authors recovered from them. They also all use JUnit (or the equivalent), but the ideas you'll pick up are readily transferable to Perl.

[adrianh]

All excellent reads.

There's also some good information online. Take a poke around testing.com and the c2.com Wiki for interesting info on testing practices.

[gjb]

Probably this is obvious to all, but this is going about the wrong way round (not that you can help it, mind). The Right Way to Do It(tm) is to first write the test, then the code it's supposed to test. At least if you buy into the extreme programming philosophy and unit testing approach.

It's better to develop tests and code in that order since at the point you write the test, it's still a black box, and you're forced to think ahead of border and corner cases. In a way, the developer is more open-minded towards his own work before he's done it than afterwards.

Again, it doesn't apply to your particular situation, but it's something that might be worth keeping in mind for future projects. And yes, it's all in the books mentioneed above, but I like to stress it nevertheless ;-)

And again, yes, I do program that way and enjoy it a lot.

Just my 2 cents, -gjb-