Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

writing tests on modules

by arcnon (Monk)
on Nov 28, 2005 at 03:27 UTC ( #512081=perlquestion: print w/ replies, xml ) Need Help??
arcnon has asked for the wisdom of the Perl Monks concerning the following question:

Does some know of a best practices primer when writing test scripts for modules. I am not sure exactly what to test for. My functions check bounds. Do I need to test it anyway. If I am returning a random array element what is an appropriate test for this type of thing? Are there times you don't need tests on a module?

Comment on writing tests on modules
Re: writing tests on modules
by sgifford (Prior) on Nov 28, 2005 at 04:32 UTC
    The usual set of test scripts are "regression tests," designed to make sure that nothing breaks when changes are made to your module or it's installed on another machine. They should test basic functionality, things that are likely to be incompatible on other systems, and things that are likely to break as you're making changes. An important part of maintaining these tests is making sure that as you fix bugs, you add tests to make sure the bugs don't come back.

    As for how thorough you want to make your tests, that's up to you. If your bounds checking is essential to your module or you think it's likely to break, check it; otherwise skip it.

    For testing random scripts, srand can be useful to make sure that rand will always return the same sequence while testing. You can also call the function you're testing several times, and make sure the results are consistent and not all the same.

      For testing random scripts, srand can be useful to make sure that rand will always return the same sequence while testing. You can also call the function you're testing several times, and make sure the results are consistent and not all the same.

      Calling the function repeatedly isn't a very robust test and using srand isn't a good idea because the underlying random number generator isn't the same across all platforms. I wrote Test::MockRandom specifically to help testing code that relies on random numbers. It overloads CORE::GLOBAL::rand to return a number from a user-specified list. This isolates the randomness from the algorithm that utilitizes the random number, allowing for robust testing.

      As part of an article for The Perl Review last year, I wrote File::RandomLine as a simple example of how to use Test::MockRandom. Looking at the tests for that module might serve as a good "cookbook" tutorial.

      -xdg

      Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

Re: writing tests on modules
by bobf (Monsignor) on Nov 28, 2005 at 04:44 UTC

    First of all, welcome to the wonderful world of testing! I recently started writing tests for my modules, and I'm hooked. :-)

    References you might find helpful:

    Relevant nodes found by Super Search (this is one of your best friends at Perl Monks):

    Finally, don't forget to search CPAN:

    Your specific example of a function that checks bounds is certainly something that I would test. It will allow you to easily verify that it is behaving as expected (per the specifications), especially after you tweak the code (anything from a minor change to a complete refactoring).

    For example: Does the function handle different types of input appropriately? What should happen if it is passed undef? An integer? A float? Does it work with positive and negative values? An alphanumeric string? A string that contains with special characters? The empty string? What other edge cases are there?

    Just some initial thoughts. Hope this helps.

      Yes, and to filter that a bit, I would start by reading the basics: Test::Simple (first) and Test::More (after).

      Test::Tutorial, though temptingly named, is unfortunately only a placeholder.

        It's true that I can't seem to find Test::Tutorial on CPAN any more, perhaps that is because it's part of the core Perl distribution since:

        perldoc Test::Tutorial

        works on the Linux boxes that I have access to.
Re: writing tests on modules
by TedPride (Priest) on Nov 28, 2005 at 07:44 UTC
    For a random array element, the important things to check are:

    1) Does the element always fall within accepted bounds?
    2) Is the distribution properly random?

    Best thing to do is set up a wrapper to run your function some large x number of times, then store counts of each element number and print out the counts sorted by element number. This will show distribution and min / max. You can also calculate the average deviation from what would be the best possible distribution. For instance:

    use strict; use warnings; my $min = 10; my $max = 100; my $tests = 10000; my (%c, $n, $average, $deviations); for (1..$tests) { $n = makerandom($min, $max); $c{$n}++; } $average = $tests / ($max - $min + 1); for (sort {$a <=> $b} keys %c) { $n = $c{$_}; $deviations += abs($n - $average); print "$_ => $n\n"; } print "Average deviation +-" . $deviations / ($max - $min + 1) / $aver +age * 100 . " % from $tests tests"; sub makerandom { my ($min, $max) = @_; return int rand ($max - $min + 1) + $min; }
    As you can see by increasing the number of tests, deviation drops to very little as the number of tests gets very large, therefore the makerandom function is working just fine.

    Average deviation +-6.9454945054945 % from 10000 tests
    Average deviation +-2.24830769230769 % from 100000 tests
    Average deviation +-0.77529010989011 % from 1000000 tests

Re: writing tests on modules
by tirwhan (Abbot) on Nov 28, 2005 at 09:25 UTC

    It helps if you think of your test suite as the most detailed specification for your modules. Any functionality your module is expected to deliver should be tested. Any failure condition your module can encounter should be tested to see whether the failure is handled correctly (for example does the module die when encountering an out-of-bounds value or does it return undef and log an error). The easiest way to do this is to write your tests for a given functionality before you write the implementation. Don't write a single line of code without first writing a test for it.

    For the example you give, returning a random array element, off the top of my head I can think of the following tests:

    • Is the returned value an element of the array.
    • Is the array changed/unchanged after you have fetched the element(depending on your requirements)
    • Is the index of the returned element properly random
    • What happens if the function parameters are out of bounds
    • What happens if the parameters are of an unsuitable type (string, floating point number)
    • What happens if the function is called without parameters
    • What happens if the array is empty

    Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. -- Brian W. Kernighan
Re: writing tests on modules
by cog (Parson) on Nov 29, 2005 at 09:51 UTC
    If I am returning a random array element what is an appropriate test for this type of thing?

    You may want to check Test::RandomResults.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://512081]
Approved by GrandFather
Front-paged by sgifford
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (4)
As of 2014-09-21 15:54 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (172 votes), past polls