Re: Re: What efforts go into a programming project? (Somewhat OT)

[Sherlock]

Well, it would appear to me that the testing portion of this process has been the part that most people seem to disagree about. Not being very knowledgeable in the realm of eXtreme Programming (XP), I'd be interested to hear from someone who is what their take on testing is.

In your post, bikeNomad, you state: you build the test code first, and then design/code until the tests pass.

Does this imply that when doing XP, you only perform Black-Box tests? What about white-box tests? These can't really be designed until you written the code, or at least done an exhaustive job on the design and know exactly what your code will look like.

Since you can't really ever perform black box or white box tests exhaustively, I feel that you really need to do a good job of both. Does XP provide for this? I only ask because your post seemed to imply that it did not.


To get back to the original post, however, I think you've gone a little overboard on the time spent doing design. Don't get me wrong, I think the more time you can dedicate to doing design, the better (more maintainable, higher quality, ect.) the product you'll develop. On the other hand, it seems like you're skimping on some other areas, specifically, coding and testing (primarily testing). You can have an exquisite design, but if you're not testing against the requirements, your final product probably won't be as good as you'd hoped.

To draw these two points (lack of testing time & black/white box testing) together, let me give you my opinion of how testing should be done.

Black Box Testing:
These should be set up when the requirements are created (just as bikeNomad mentioned). You should know, even before creating your project, what it should produce given various inputs, which is the entire nature of black box tests.

White Box Testing:
Your white box tests should be designed to ensure complete path coverage (ensuring that every line of code is executed). This can't be done until your software has been written. This is very important because you can't possibly test your program against all inputs using Black Box Testing. This technique is helpful in finding different types of errors.

Now that I've rambled on long enough, I think your assumptions look pretty good. I'd add a little time to testing, however. I hate to take time away from design, but I don't think you can really take any more away from anywhere else. Hopefully, this will help you.

Good luck,
- Sherlock

P.S.
For anyone that is unfamiliar with the terms Black/White Box testing: http://www.scism.sbu.ac.uk/law/Section5/chap3/s5c3p23.htmlgo here. This was just the first site I found - I'm sure you could find better ones if you looked a little.

Skepticism is the source of knowledge as much as knowledge is the source of skepticism.

[bikeNomad]

At first, you can only do black-box tests (since, as you point out, there's no code yet). There's certainly room for white-box testing in XP, but it would come later. After all, since your team designed it, coded it, and wrote the tests, tests added after some code has been written can't help but be anything but white-box!

I think a lot of the distinction between white-box and black-box testing assumes that people writing/performing tests will be different than the people designing/coding. With XP, of course, it's the same people making and running the unit tests.

Coverage analysis would help to find what you've missed tests for. It's hard to err by having too many tests.

In XP, you keep adding tests for new features/behavior before you add the new features themselves. At that time, you could add more tests for boundary cases in the code you just wrote if you think it's going to be a problem.

[carlfish]

In XP, there are two kinds of tests - "Acceptance Tests" and "Unit Tests". The Acceptance tests ensure that the code conforms to the requirements, end-to-end. The Unit tests are written against each individual part of the code as it is being developed. You also take significant steps to isolate the code that is being tested from the rest of the program.

Unit Tests are mostly white-box in that they are written in conjunction with the source-code. The first test proves that you can't get away with just doing nothing. After you have written the first test, you will examine your code, and write a second test that illuminates a failure condition within that code. You then work until that test ceases to fail, and continue the cycle until you can't think of anything more to test.