Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

Testing for Beginners

by markmoon (Deacon)
on Apr 08, 2004 at 12:57 UTC ( #343595=perlquestion: print w/ replies, xml ) Need Help??
markmoon has asked for the wisdom of the Perl Monks concerning the following question:

Lately, it seems that many people have begun to champion the benefits/necessity of building tests. As someone who doesn't have the opportunity to write Perl for a living, it means I also don't have the added plus of working with other people to see how test-driven projects are built.

So far my only exposure has been a Portland Perl Mongers talk by Brian Ingerson which was very good and a seminar given by a leader in XP that seemed kind of disjointed to be honest. At the time I wasn't ready for either. Can anyone point me towards good examples that can help explain the core concepts and possibly contain tutorials that don't assume previous experience building large apps?

Thanks,
That pesky markmoon guy
--
@a = ("a".."z"," ","-","\n");foreach $b ( 12,0,17,10,24,12,14,14,13,26,8,18,26,0,26, 22,0,13,13,0,27,1,4,26,15,4,17,11,26,7,0, 2,10,4,17) {print $a[$b]};print $a[28];

Comment on Testing for Beginners
Download Code
Re: Testing for Beginners
by halley (Prior) on Apr 08, 2004 at 13:24 UTC
    You don't have to be "test oriented" to start reaping the benefits of using tests.

    Some people use tests to drive design. This is what I would call "test oriented." They first think what the program should do, then they write tests to see if it does it (and doesn't do anything else), then they write the code to do it. In the process of writing the tests, they refine their design before they even get to the actual code. Downside: you have spent a lot of coding time before you get any results, and you scrap a lot of tests as you revise your design.

    Some people use tests to drive execution. This is often called "programming by contract." In each subroutine, tests are embedded to ensure that design-time expectations meet run-time reality, at entry and at exit. Downside: performance can suffer, and your subroutines start to look like that old public-speaking maxim: "first you say what you're going to say, then you say what you came to say, then you say what you just said."

    You can use tests to drive collaboration and deployment. These are often called "smoke tests." Where there's smoke, there's fire. This is probably the most common in practice. Smoke tests are automatable, and smoke tests can be run without any other knowledge of the application. Include a few test scripts with your new code, and reference those tests in your bug reports. Run the tests before you check in changes to your repository. Run the tests every night by daemon and email the results. Run the tests again whenever you upgrade your computer. Run the tests again when a new operating system is released. Run the tests whenever you are idle. The downsides are, well, you have to actually run the tests, and it's harder to ensure the tests are actually providing meaningful coverage.

    --
    [ e d @ h a l l e y . c c ]

      The downsides are, well, you have to actually run the tests, and it's harder to ensure the tests are actually providing meaningful coverage.
      If you are at all worried about coverage, try Devel::Cover, it is amazingly helpful. And my personal recommendation is not to read the documentation that comes with it first as it just confuses if you are unfamilair with code coverage tools (as I was). Instead, i recommend just installing it, and running it on one of your test scripts, once you see the output, it will all make a lot more sense.

      -stvn
Re: Testing for Beginners
by bronto (Priest) on Apr 08, 2004 at 13:45 UTC

    I started using the test suites of Perl, especially Test::More, after reading Sam Tregar's Writing Perl Modules for CPAN, which I reccomend. Testing is a small part, but there is a very good Perl book around it! :-)

    Ciao!
    --bronto


    The very nature of Perl to be like natural language--inconsistant and full of dwim and special cases--makes it impossible to know it all without simply memorizing the documentation (which is not complete or totally correct anyway).
    --John M. Dlugosz
Re: Testing for Beginners
by Art_XIV (Hermit) on Apr 08, 2004 at 13:45 UTC

    There are some links to some good perl testing docs here.

    You may also be able to get some more background/philosophy from www.junit.org and www.xprogramming.com, although these sites are very XP-oriented and have little that applies specifically to Perl and the Test:: family of modules.

    Hanlon's Razor - "Never attribute to malice that which can be adequately explained by stupidity"
Re: Testing for Beginners
by Ovid (Cardinal) on Apr 08, 2004 at 13:48 UTC

    Even if you don't write the tests up front, you can still benefit. I once gave a testing talk to a crowd of programmers who used C#. I mentioned CSunit and one of the programmers just took off and started using it on a large app that he had built. Several days later, he mentioned to me that he had found several long-standing bugs that he never knew about (in my experience, this is common for people starting testing).

    I'd start with Test::Tutorial and just start writing a few small tests for something you've worked on. That should get you the feel for it. If you're programming in another language, check out testing tools for your language. (They even have them for Javascript).

    Cheers,
    Ovid

    New address of my CGI Course.

      I started going through Test::Tutorial and decided to write some better tests for one of my modules on CPAN. Wouldn't you know it, it exposed a bug in a section of code that I wrote but probably never had the reason to use. Guess it's time to collect all those improvements and submit a new version!

      90% of every Perl application is already written.
      dragonchild

        Heh :) I hear that all the time. My favorite was when I started writing tests for a security module and immediately found a transient bug that had been stumping me for around a year!

        Once you get into testing, check out Devel::Cover. It is probably the single most important testing tool outside of the actual tests.

        Cheers,
        Ovid

        New address of my CGI Course.

Re: Testing for Beginners
by blue_cowdawg (Prior) on Apr 08, 2004 at 14:30 UTC

        Lately, it seems that many people have begun to champion the benefits/necessity of building tests.

    For me there is no "lately" about it. I have always championed the concept of testing suites for software for anything more complicated than a one liner.

    The methodology that I use for testing is a simple one. I ask myself the following quesitons:

    1. What is the base functionality of this software?
    2. How do humans interact with it?
    3. How can I simulate that?
    4. How can I simulate the interactions going wrong?

    Once I've answered those questions for myself I proceed to develop a test plan and that most of the time will drive how I write my test suite.

Re: Testing for Beginners
by Anonymous Monk on Apr 08, 2004 at 15:32 UTC
    As a noob, I am in favour of the test first strategy, I think it can/will save me lots of time in the long-run if I can teach myself to do it! I am primarily writing web apps, CGI.PM and DBI (using DBD::ANYDATA) so some guidance on writing tests for this sort of thing would be appreciated. Below is some code I threw together this afternoon to create some CSV datafiles using SQL Perl, etc etc. It works, I know this because I tested it manually. How would I write a proper test for this? And yes I know I should have written the test first, but lets ignore that for now. Any other constructive criticisms would also be appreciated.
    #!/usr/bin/perl -wT # --------------------------------------------- # # Description: # This script is designed to be used once to install the system # We will use DBI and DBD::AnyData to create CSV datafiles # We use CGI.PM to handle HTML # # History: # ======== # 07 April 2004, - Initial file created, based on e-judo.cgi at e-judo +.sourceforge.net # 08 April 2004, - Wrote the subs to create all the data files. my $DEBUG = 1; # If this is set to 1 then we see the debug messag +es. use strict; # force strict programming controls use CGI qw(:standard); # use the CGI.PM module use lib './MyLib'; # use the modules in MyLib, this is the DBD::Anydat +a used for database activities use DBI; # This calls the DBI module, which along with the line above +allows us to do database activities # Sub Routines # ---------------------------------------------------------------- sub create_users_file { # This sub creates the users_csv file print p("Start of create_users_file") if $DEBUG; # create the scalers we need to use in the sql # ---------------------------------------------- my $table = "data/users_csv/"; print p(" table name = ", $table ) if $DEBUG; # Okay now we must create the database files # here is the DBI/SQL code # ---------------------------------------------- +------ # First create the array and hash to hold the ta +ble fields and data definitions # ---------------------------------------------- +-------------------------------- my @table_fields = qw/ user_id first_name last_n +ame email credits login_id password /; my %table_field_def = ( user_id => 'char(20)', first_name => 'char(20)', last_name => 'char(20)', email => 'char(20)', credits => 'char(20)', login_id => 'char(20)', password => 'char(20)' ); print p(" Table Fields = ", @table_fields) if $ +DEBUG; print p(" Table Fields def = ", %table_field_de +f) if $DEBUG; my $dbh = DBI->connect('dbi:AnyData(RaiseError=> +1):') or die "Can not create database connection"; # build the table using SQL # --------------------------------- $dbh->do ( "CREATE TABLE the_table (" . join(',', map { $_ . ' ' . $table_fiel +d_def{$_} } @table_fields) . ")" ) or die "Can not create table"; $dbh->func( 'the_table', 'CSV', $table +, 'ad_export'); print p("User table created") if $DEBUG; print p("END of create_users_file") if $DEBUG; } # End Sub-Routines # ------------------ # Main Code starts here # ----------------------- print header(), start_html("Installation"), h1("Install Script"); # Th +is line uses CGI.PM to to create the webpage if (param()){ # If there is a parameter(or parameters) then validate, + else show the login screen. # the following lines are excecuted if paramaters HAVE been entered my $confirm = param("confirm"); # $confirm is the text en +tered on the webpage form entered by the user if ($confirm eq "YES") { # If the user enetered YES +(in caps) then run the install # first, check if the users dataf +ile exists, if not we will create it. if (-e "data/users_csv"){ print p(" +users_csv exists") if $DEBUG; } else { print p +("users_csv does not exist so about to call the create_users_file sub +") if $DEBUG; create_us +ers_file(); } } # end if if statement for $confirm } else { # if there no parameters print a webform and ask conformati +on to install print hr, start_form; # create a form using CGI.PM print p("Please type in YES (In capitals) to proceed with i +nstallation: ", textfield("confirm")); print submit(-name=>'submit button'); print end_form, hr; +# end the form } print end_html; # this closes the web page properly
      Sorry folks, somehow ended up logged out when I posted that.

      Kia Kaha, Kia Toa, Kia Manawanui!
      Be Strong, Be Brave, Be perservering!

        Some quick 'n dirty testing guidelines:

        • It's easier to test modules than scripts. Move the code to be tested into modules.
        • It's usually poor design to have a function or method do too many things. Coincidentally, it makes them harder to test. Break them into separate functions.
        • For scripts, I frequently have a DEBUGGING constant. If it's true, I have it runs tests embedded directly in the code (it's easier if you can run the CGI script from the command line). You can see a simple example of embedding tests (without the constant) at Why is goto &sub slow?.
        • You can also mock up troublesome interfaces (if they're tested separately!) by using Test::MockObject or something similar (I frequently localize the sub and have it return results directly).

        You might also be interested in Test::AtRuntime. It's a great module that can solve some of these problems.

        Cheers,
        Ovid

        New address of my CGI Course.

Re: Testing for Beginners
by markmoon (Deacon) on Apr 08, 2004 at 15:37 UTC

    Thanks to everyone for the recommendations and links.

    markmoon

    --
    @a = ("a".."z"," ","-","\n");foreach $b ( 12,0,17,10,24,12,14,14,13,26,8,18,26,0,26, 22,0,13,13,0,27,1,4,26,15,4,17,11,26,7,0, 2,10,4,17) {print $a[$b]};print $a[28];
Re: Testing for Beginners
by xdg (Monsignor) on Apr 08, 2004 at 17:12 UTC

    chromatic has given some talks on testing in perl. Slides to various talks (testing and otherwise) are available from http://www.wgz.org/chromatic/talks/

    Also, in my own quest for knowledge, I discovered prove is installed as part of my perl distribution (5.8, maybe others), which lets you specify particular test files to run. It looks quicker than running "make test" and setting various command line variables for make when you just want to test a single (or a few) *.t files. "man prove" for details.

    -xdg

    Code posted by xdg on PerlMonks is public domain. It has no warranties, express or implied. Posted code may not have been tested. Use at your own risk.

Re: Testing for Beginners
by mutated (Monk) on Apr 08, 2004 at 17:13 UTC
    I'm a big fan of writting tests after a program (or function) is written to test the functionality of that function and tie it into the make test product, I work with a team of people an no one "owns" the code, the tests are there so that if someone modifies the code and my tests still pass, it is likely they haven't broken anything important. You can never catch everything with tests (the tests you forgot to write probably would have checked the exception you forgot to handle in your code) but at least when you come across an exception you can write a test for it, so that future versions of your software will never again fail due to that exception.


    daN.
Re: Testing for Beginners
by flyingmoose (Priest) on Apr 08, 2004 at 19:13 UTC
    It is important to be very honest about writing your tests and understanding what they do.

    Basic sanity of compilation and basic functionality can be automated, but often things like complex usage, race conditions, network conditions, or tools that interoperate with other components cannot be accurately tested.

    Thus, automated (aka computer-run) testing helps the programmer but it can never assure perfection in the code in non-trivial cases. Use caution. Basic tests are a good idea, but it is important to fully understand what either (A) your tests don't cover yet or (B) what it is infeasible for you to automate. Use caution especially around XP advocates, who are known for writing tests such as "verify 1 + 1 = 2". Ok, I exaggerate, but testing for seeming tautologies (while sometimes important), does not good testing make.

    I still, to this day, prefer reading code, understanding it, and testing by hand. That may be because I work with pathologically untestable software, but it's also based on a need for caution and a higher level of testing.

      Use caution especially around XP advocates, who are known for writing tests such as "verify 1 + 1 = 2".

      While I confess that there are many XP advocates who get down to that level of detail, what you should hear from them is "don't test what can't fail". I would never test that chomp is doing its job, for example. However, the flip side of that is important. When something that can't fail does, then you write the test.

      Cheers,
      Ovid

      New address of my CGI Course.

        If you want to buy into six degrees of seperation, the XP-type people I have known, met, or heard and had problems with either their thoughts or their examples include:

        A professor/consultant, who was good friends with Kent Beck and involved in the books (SEP: 1) -- he advocated code should average 2.5 lines per method, which is utter hogwash and results in just a different kind of maintaince problem -- which I have oft referred to as politically correct obfuscation. He essentially took the obfuscation inherit in Java's OO model when overblown, and overblew it further.

        Another professor, who was a follower of that guy above (SEP: 2)

        Various other professors who were fairly hardcore on the subject (SEP: 3) and also advocated similarly lame XP testing practices

        Martin Fowler (SEP: 0) (but only on the controversially lame "refactoring" subject, which mostly stated the obvious)

        Anyhow, beware the academics who do not practice! And beware the overpaid consultants as well! "Best practices" is always a phrase that must be evaluated carefully and with a rationale mind. Keep asking "well, it sounds good, but does it really work...and why?"

        Don't get me wrong, testing can work...but is part of a whole, and I definitely don't buy into XP-based test-driven development, pair programming, puny methods, no design up front, or ... well ... any of it. That being said, I *AM* a advocate for pristine clean code, quality software, and good testing. I'm not slacking here, I'm saying don't buy into the hype, XP doesn't solve as many problems as they claim. And it clearly isn't for everyone.

Re: Testing for Beginners
by stvn (Monsignor) on Apr 08, 2004 at 21:43 UTC
    markmoon,

    Everyone here has given some great advice, but for me the testing lightbulb went off when I first really saw and messed around with an existsing test script. It was always too easy for me to put off reading tutorials and such (yes, the wrong kind of laziness), but when i finally got in front of a test that was already set up and ready to go, it all clicked. So in the spirit of passing on what worked for me, I have mocked up a quick test script that might help you get your feet wet.

    First things first, you need to download and install the latest versions of Test::Simple (which has Test::More in it, which is your workhorse), and Test::Harness. You more than likely already have both on your system, but it can't hurt to get them up to date, and plus, Test::Harness comes with this great utility script called "prove", which makes running tests a lot simpler.

    Now, here goes the code, it is purposely VERY simple so as not to cloud the issue, which is testing, with funky code.

    Put this code into a file called "Module.pm"

    package MyModule; use strict; use warnings; sub add { my ($left, $right) = @_; return $left + $right; } 1; __END__

    Put this code into a file called "test.t"

    #!/usr/bin/perl use strict; use warnings; # we have 3 tests to run use Test::More tests => 3; # if you dont know how many tests # you have you can just replace: # tests => 3 # with: # "no_plan" # that way you dont have to count # your tests, although when you are # finished its best to put the plan back # in place. # test that we can require the module require_ok("MyModule.pm"); # it also has the added benefit of requiring # the actual module # test that the module has the function "add" available can_ok("MyModule", 'add'); # now test "add" works as expected cmp_ok(MyModule::add(1, 1), '==', 2, '... 1 added to 1 is 2');

    Now make sure both files (MyModule.pm and test.t) are in the same directory, and then run this from the command line (assuming you have already cd/chdir to the same directory).

    prove test.t
    You should see the following output:
    test....ok + All tests successful. Files=1, Tests=3, 0 wallclock secs ( 0.15 cusr + 0.05 csys = 0.20 C +PU)
    And thats all. Obviously a more complex module would require a more complex tests, and take note, my tests are very simplistic. But this might give you help in getting started, as it helped me.

    -stvn
Re: Testing for Beginners
by tphyahoo (Vicar) on Jun 28, 2005 at 13:47 UTC
    Found a very nice Test Harness Tutorial: Minimal Testing With Test Harness.

    I'm copying the contents into perlmonks in case the website disappears some day.

    Minimal Example for Test::Harness The Test::Harness module is part of the standard perl distribution. It + is widely used for testing modules. Learn to use the module by study +ing this simple example. Mymodule.pm A very silly module to test. Here it is: package Mymodule; sub Mysub { return "This is my expected output\n"; } 1; Myapp.pl Silly use for a silly module. #!/home/toma/perl5i/bin/perl use strict; use Mymodule; my $result= &Mymodule::Mysub(); print $result; tst.pl This program calls the tests. Some applications read the contents +of the t directory and call all the tests there. That way you don't h +ave to keep updating this file when you add a new test. #!/home/toma/perl5i/bin/perl use strict; use Test::Harness; my @tests; push @tests, qw ( t/tst1.t t/tst2.t ); runtests(@tests); t This is the directory where the tests are traditionally located. E +ven though they are perl programs, they have the file extension '.t'. t/tst1.t This test doesn't really test the module, but it shows you how a t +est is expected to succeed. use strict; print "1..2\n"; print "ok 1\n"; print "ok 2\n"; t/tst2.t This test actually looks at what the module returns to see if it i +s correct. Test programs such as these can print all sorts of extra g +arbage. The harness just looks for a few regular expressions that sho +w success or failure. #!/home/toma/perl5i/bin/perl use strict; use Mymodule; my $result= &Mymodule::Mysub(); print $result, "\n"; print "You can print all kinds of garbage in your test\n"; print "1..2\n"; if ($result =~ /This is/i) { print "ok 1\n" } else {print "not ok 1\n"} if ($result =~ /expected/i) { print "ok 2\n" } else {print "not ok 2\n"} print "Anything that makes sense at the time\n"; t/tst3.t This one fails the second and third tests. #!/home/toma/perl5i/bin/perl use strict; use Mymodule; my $result= &Mymodule::Mysub(); print $result, "\n"; print "You can print all kinds of garbage in your test\n"; print "1..3\n"; if ($result ne "") { print "ok 1\n" } else {print "not ok 1\n"} if ($result =~ /expected-but-not-there/i) { print "ok 2\n" } else {print "not ok 2\n"} if ($result ne "only the good stuff") { print "not " } print "ok 3\n"; print "Anything that makes sense at the time\n"; Makefile This allows you to use the command make test and have the right th +ing happen. test: perl tst.pl make test This is the output from make test. perl tst.pl t/tst1..............ok t/tst2..............ok t/tst3..............FAILED tests 2-3 Failed 2/2 tests, 0.00% okay Failed Test Status Wstat Total Fail Failed List of failed ---------------------------------------------------------------------- +--------- t/tst3.t 2 2 100.00% 2-3 Failed 1/3 test scripts, 66.67% okay. 1/6 subtests failed, 83.33% okay +. make: *** [test] Error 29 Fixing the @INC Path Sometimes a test may have trouble loading the module that you are tryi +ng to test. To fix it, add this code to the tests, for example in t/t +st1.t. use FindBin; use lib "$FindBin::Bin/.."; This has the effect of adding the relative path ".." to the INC variab +le at compile time. You might hope that the code use lib ".."; would +work, but it doesn't. Ordinary use lib "/path/to/my/file"; commands a +dd to the INC path properly, but can't handle relative paths. Conclusion You can learn to enjoy the benefits of developing tests concurrently w +ith your code. If you create a new test for bugs that you or others f +ind, you are really doing great work!
    Update: For a followup... Is there a better way to get all the t/.t test files an array?.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://343595]
Approved by broquaint
Front-paged by bronto
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (6)
As of 2014-08-23 06:00 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (172 votes), past polls