Re^6: Neither system testing nor user acceptance testing is the repeat of unit testing (OT)

in reply to Re^5: Neither system testing nor user acceptance testing is the repeat of unit testing (OT)
in thread Neither system testing nor user acceptance testing is the repeat of unit testing (OT)

I'm not limiting myself to exceptions and throws - I'm talking about exception flows - parts of the cod that gets executed when something goes wrong. This may result in an exception getting thrown, but only if the designer (probably not me) says I can throw - should the design be at that level of polish. Maybe the designer says - thou will not throw an exception in your implementation - I might disagree, but thats what I _have_ to do. I dont think I'd have a job long if I said "screw you , I'm throwing one anyway."

What I am referring to are testing the decision I made for the implementation of the design. _If_ I chose to use an on-disk file as part of my solution, and _if_ I wrote code to handle different I/O failures, I would want to make sure I wrote unit test cases to check I correctly implemented handling the different failures. Maybe I just handle them and dont tell the caller anything, maybe I log it and continue on. Whatever I chose to do, I should test I implemented correctly. How wise my on-disk solution is, that's a different question.

In short, those things I can test from the interface/design point-of-view, I agree with you completely and I should test black-box, those things that are artifacts of my implementation, but not part of the interface/design, I should test black-box if I can, and white box where I need to. Its all about making sure you tested "everything" - well it is for me, unit testing my code to as close to 100% coverage as possible is what I aim for. In the OP's case, calling a bad SQL wasnt something their testing had caught. Somewhere, if they ran a coverage report, they might see where that SQL was being injected, as a piece of code that hadn't been tested - a very white box approach.

...reality must take precedence over public relations, for nature cannot be fooled. - R P Feynmann

Comment on Re^6: Neither system testing nor user acceptance testing is the repeat of unit testing (OT)

Replies are listed 'Best First'.
Re^7: Neither system testing nor user acceptance testing is the repeat of unit testing (OT) by Anonymous Monk on Oct 24, 2005 at 23:39 UTC
Much as I typically disagree with dragonchild, I'll agree with him on this point. Your caller should know what the impact of calling your function is. If it has side effects or dependencies (such as requiring sockets to be available, stomping on global variables, requiring filesystems to be writeable, or so forth), those restrictions should be published as as part of the function interface, so that they can be tested, by someone other than the author of the code. If a tester can get an undocumented side effect from calling a given function, that's a bug in the code, or the code documentation. Either is bad. How much this matters depends on what you're dealing with, and what failure modes you can handle. If, for example, you're sure your production server can't possibly run out of filehandles, because it will crash due to other limitations long before that becomes an issue, perhaps there's no real viable way to test, let alone recover from the "out of filehandles" condition. If you document your position, it's at least a calculated risk, not just an accidental assumption, and if it proves to be false, you at least have a explanation for why the choice to ignore it was made. And if/when you upgrade your server, you'll catch the limitation of the software when you read through the limitations list for all your server code prior to the upgrade. -- AC	[reply]

Replies are listed 'Best First'.

Re^7: Neither system testing nor user acceptance testing is the repeat of unit testing (OT)
by Anonymous Monk on Oct 24, 2005 at 23:39 UTC

Your caller should know what the impact of calling your function is. If it has side effects or dependencies (such as requiring sockets to be available, stomping on global variables, requiring filesystems to be writeable, or so forth), those restrictions should be published as as part of the function interface, so that they can be tested, by someone other than the author of the code.

If a tester can get an undocumented side effect from calling a given function, that's a bug in the code, or the code documentation. Either is bad.

How much this matters depends on what you're dealing with, and what failure modes you can handle. If, for example, you're sure your production server can't possibly run out of filehandles, because it will crash due to other limitations long before that becomes an issue, perhaps there's no real viable way to test, let alone recover from the "out of filehandles" condition. If you document your position, it's at least a calculated risk, not just an accidental assumption, and if it proves to be false, you at least have a explanation for why the choice to ignore it was made. And if/when you upgrade your server, you'll catch the limitation of the software when you read through the limitations list for all your server code prior to the upgrade.
--
AC

[reply]

In Section Meditations