http://www.perlmonks.org?node_id=982250


in reply to Re: Universal test flag
in thread Universal test flag

If code needs to know that it is being tested, I suspect something wrong with the interface.

Probably so, and in theory everything sound nice, but in real-world every-day coding, with a team of coders, and with time and budget constraints, this is by far the common case. You need to be able to test things while skipping others that don't concern that particular aspect that you are working on or testing.

Example: you are testing a class that invokes an external service to send an SMS, and the SMS gateway has no test mode, so it costs you money. Your class does a hundred things besides sending the SMS and has a lot of business logic and you only want to skip the SMS sending part to test this logic. Maybe the SMS sending logic was encapsulated in a class (in which case could be mocked) or maybe it's part of the same class on a sub and can't be mocked. Like this one, there are a many day-to-day scenarios where you want to skip over some part of the code just o test the overall logic.

Refactoring to make code perfectly testable is not always an option, in fact, it rarely is, at least in large projects. The code is far from perfect and you have to make the best of it and try to test it as best you can within the time and budget constraints that a particular project allows.

Replies are listed 'Best First'.
Re^3: Universal test flag
by chromatic (Archbishop) on Jul 17, 2012 at 17:02 UTC
    Refactoring to make code perfectly testable is not always an option...

    ... but writing more code is an option?

    If you have to write extra code in the code you're testing that only gets executed in test mode, I think your confidence in the test results goes down.

Re^3: Universal test flag
by zwon (Abbot) on Jul 17, 2012 at 18:25 UTC
    Refactoring to make code perfectly testable is not always an option, in fact, it rarely is, at least in large projects

    The purpose of refactoring is to make code structure optimal, testability is just a side effect. And it is especially important for large projects, without maintaining code structure you will soon finish with spaghetti which will take years to unravel. Particularly, if you have classes which do hundred things and send SMSes, it does sound alarming.

Re^3: Universal test flag
by ait (Hermit) on Jul 18, 2012 at 13:42 UTC

    chromatic and zwon:

    Obviously your world seems more perfect than ours, but the hard truth in the commercial world is that there is always a tradeoff between quality and the amount of money and time available. Most of the projects today are constrained in both, and you must quickly adapt constantly changing business conditions, so refactoring always falls in second place, after you get the (ever changing) functionality pinned down.

    Testing on the other hand is paramount to make sure each piece does what it should and all the pieces fit together when several people are working on the same project. Of course, code must be good enough to be tested but it surely doesn't have to be perfect.

    Sure, software is perfectible if you have enough time and resources, but the cold hard truth is that in most situations customers are not willing to pay the extra money for perfect code and their budget only allows for "good enough". IMHO, the truly successful business projects are able to deliver in time and money with good enough code to get business flowing and creating the cash flow necessary to eventually perfect the code.

      Obviously your world seems more perfect than ours, but the hard truth in the commercial world is that there is always a tradeoff between quality and the amount of money and time available.

      I've been in management for five years now. I've been doing automated testing for fourteen years now.

      Now that we're past the credential waving protocol, let me paraphrase what I said, lest it be waved away under the flag of "We don't have time to do it right. We're doing Serious Business".

      Of course, code must be good enough to be tested but it surely doesn't have to be perfect.

      If you run your code differently under testing than you do in deployment, your tests probably aren't testing what you care about.

      I didn't use the word "perfect". Please don't read into what I wrote; you'll confuse things.

        Yeah, I should have answered separately because I was mostly answering to zwon, I apologize for that, but you shouldn't have turned this into a pissing contest either.

        If you run your code differently under testing than you do in deployment, your tests probably aren't testing what you care about.

        I understood you the first time around, but it's quite common to have several tests for the same piece of code, or have a test behave differently given certain test conditions (static, offline, live, etc.). More commonly this is done with skip blocks in the test code, but sometimes you don't have that option.

        I found tobyink's approach the best because it would even allow to run the app for integral testing with certain parts shut-off. I don't see this any different that skipping tests from the test suite.

      the hard truth in the commercial world

      Oh, you apparently think that I'm living off charity and only write abstract examples ;)

      refactoring always falls in second place, after you get the (ever changing) functionality pinned down

      refactoring is not something you do after implementing functionality, it is something you do to implement functionality in a most efficient way (and so save time and money)

        Oh, you apparently think that I'm living off charity and only write abstract examples ;)

        No sir, I don't make those assumptions ;-)

        What I'm saying is that it's very different, for example, to code a well thought-out CPAN library, than working on a large system with a team of coders at different levels, with time and money constraints.

        I think that your comments refer more to design than to the process of coding. Refactoring by very definition is a continuos and disciplined process after the fact, so your assumption above is actually wrong and must not be confused with good design. Good design is paramount, and that is not in discussion.

        Furthermore, to actually be able to refactor you must first have a solid set of tests so when you actually refactor, you can guarantee the same functionality that should have been pinned down beforehand.