Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask
 
PerlMonks  

Re^5: Neither system testing nor user acceptance testing is the repeat of unit testing (OT)

by dragonchild (Archbishop)
on Oct 24, 2005 at 13:17 UTC ( #502451=note: print w/replies, xml ) Need Help??


in reply to Re^4: Neither system testing nor user acceptance testing is the repeat of unit testing (OT)
in thread Neither system testing nor user acceptance testing is the repeat of unit testing (OT)

This is so funny. I feel that your argument against my point only ends up supporting it. "corner cases and special cases in the code" are very much a common source of bugs. However, I think that having corner/special cases is a CodeSmell. (Yes, I'm being idealistic here - there is no application whose code smells like roses.)

Plus, imho, Devel::Cover is a measurement of the success of your black-box tests vis-a-vis your interface. If you have lines of code you don't have tests for, that's a problem that needs to be addressed. Granted, you might address is by looking at the code to see which interface components either haven't been tested or have been over-coded for. But, that doesn't mean the test suite has necesssarily become white-box. I often use coverage to both learn more about my interface and to remove code that shouldn't have been there in the first place. Still black-box.


My criteria for good software:
  1. Does it work?
  2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?
  • Comment on Re^5: Neither system testing nor user acceptance testing is the repeat of unit testing (OT)

Replies are listed 'Best First'.
Re^6: Corner cases are not always "code smell"
by Ovid (Cardinal) on Oct 24, 2005 at 17:59 UTC

    I think that having corner/special cases is a CodeSmell.

    That depends upon what those corner cases are. Sometimes it is impossible to avoid them because the real world does not behave the way we want it to. A common example would be leap years. You have to have corner cases there. In fact, there are a lot of corner cases in date handling due to how tricky it can be.

    Also, hourly employees generally get time and a half if they work more than a set number of hours per week, but for some states/countries, they can also get time and a half if they work more than a set number hours per day. Those are exceptions to the rule and, as such, may get special treatment in the code.

    Or how about removing data from a database? Many systems merely mark the data as "deleted" buy don't actually delete it so that it can be recovered. However, there may be legal reasons why the data must be deleted (for example, contractual agreements). This may come up as a "special case."

    Many issues which may be a corner case can be developed in such a way that they're not a corner case (deleting database records, for example), but whether or not it's practical to do so depends upon the needs of a given application.

    Cheers,
    Ovid

    New address of my CGI Course.

      Code Smell:
      Note that a CodeSmell is a hint that something might be wrong, not a certainty. A perfectly good idiom may be considered a CodeSmell because it's often misused, or because there's a simpler alternative that works in most cases. Calling something a CodeSmell is not an attack; it's simply a sign that a closer look is warranted.

      Caution: Contents may have been coded under pressure.
      While Roy is certainly right in the definition of CodeSmell, I'd also like to point out that those corner cases are corner-cases in the spec, not the code. Reality has corner cases, code doesn't. If you run into one (leapyears, overtime, data retention), that needs to be documented in the spec and the tests first, then in the code. Except, once it hits the code, it's not a corner case - it's a requirement.

      That may sound like semantics, but I think it's an important distinction.


      My criteria for good software:
      1. Does it work?
      2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?
        Geez, you've sure drunk the kool-aide, haven't you?

        You say that, Reality has corner cases, code doesn't. But in the real world there are cases where people need optimizations. (Less than most people think, but it does happen.) Optimizations naturally lead to having corner cases. Those corner cases should generally not be visible in the interface. Therefore code sometimes does need corner cases that aren't part of reality.

        Now you may think that we should never optimize. But just today I wrote about an example where not having the optimization leads to bugs. Underscoring my point even more, that optimization should be invisible in the interface, and the initial implementation of this optimization had bugs in its corner cases. (Interestingly, the corner cases where it had bugs would not have been visible to a code coverage test like Devel::Cover, underscoring even more the need for white-box tests on it!) Therefore if your ideal is fewer bugs, then we have a concrete case where you really want to have corner cases that are not visible in the interface.

        In short, you're being blinded by your fanaticism. Real code sometimes does have corner cases that aren't visible in the interface. Those corner cases should either be removed or tested with white-box tests. I agree that often it should be removed, but you go too far when you say that they should always be removed.

        I think Ovid gave perfect examples to disprove his point. Those are all corner cases that need to be in the spec and so could be subject to black-box testing.

        However, I don't agree with the extreme point of view that every corner case needs to be in the spec. Because... You Are Not Going To Need It.

        If you spend all of your time gold-plating and turd-polishing your spec, then you run into the same problems with doing that to your code.

        We save a lot of time by leaving a lot of corner cases unspec'ed because they are implementation details and whether X or Y happens in that corner case is not going to matter for 99.9% of them. The 0.1% of cases where a corner case that was found during implementation mattered in the finished product are caught in other phases of testing or based on judgement. Most code corner cases don't need the attention of revisiting and rehashing the spec.

        But we test those corner cases in our unit tests. Usually we either want the corner case to blow up (so we know when it becomes a case that matters) or we want to make sure it doesn't blow up (because the outcome doesn't matter but you want to know that the code doesn't contain a stupid and fatal mistake just in case the "impossible" isn't as impossible as first thought).

        Corner cases do not always mean extra code. If the only corner cases in code were places where you could just remove the code for that corner case because You Are Not Going To Need It, then you could almost make your high-ground claim of "no corner cases unless in the spec" (other than optimizations, as tilly pointed out).

        That is why languages have "undefined" behaviors. Because it is a huge waste of time to over-specify every tiny detail of behavior such that you've completely tied up the hands of the implementors for the sake of guarantees that don't matter (except they matter now because your implementation sucks because it is forced to meet the micro-managing spec).

        And yet, part of the spec is that the language never dumps core. So you need to test every line of code even though some of those lines were a means (implementation detail) to an end (spec'd requirement).

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://502451]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (6)
As of 2019-11-19 07:32 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Strict and warnings: which comes first?



    Results (94 votes). Check out past polls.

    Notices?