Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Testing - General Questions

by TomKane (Beadle)
on Nov 21, 2007 at 13:24 UTC ( #652125=perlquestion: print w/ replies, xml ) Need Help??
TomKane has asked for the wisdom of the Perl Monks concerning the following question:

I am trying to understand why there is such a thing as a .t file, as opposed to just using the .pl files. Are the .t files really the same as the .pl files but with embedded tests?

When I "use" Test::Simple, Test::More, etc., in a .pl file and run it with Perl it produces test results. Likewise, running that same .pl file with prove will produce test results although the output appears different. What's the advantage of using prove? Why not simply use Perl?

Are tests embedded into production modules? If not, how does one test (verify) that the act of physically removing tests from production code didn't accidentally interject some errors?

I understand that the use of t/ directories to contain .t files is a convention for normal module installation. My problem is that I am coming from the Win32 environment using the ActiveState installation and I therefore rely on ppm for my installs. I don't get to do the "make test" part of the installation so perhaps I just don't have a full appreciation of what testing does. But I do want to have that understanding as I am working on some code I would like to distribute. I am sure development will go better if I can embed testing in it.

Thanks for any answers.

Comment on Testing - General Questions
Re: Testing - General Questions
by jbert (Priest) on Nov 21, 2007 at 14:11 UTC
    I am trying to understand why there is such a thing as a .t file, as opposed to just using the .pl files. Are the .t files really the same as the .pl files but with embedded tests?

    .t files are perl files. They have a different suffix to identify the fact they are tests, not production code.

    When I "use" Test::Simple, Test::More, etc., in a .pl file and run it with Perl it produces test results. Likewise, running that same .pl file with prove will produce test results although the output appears different. What's the advantage of using prove? Why not simply use Perl?

    We do use perl. The different extension simply indicates to the user or test harness (e.g. prove) that this is a test which should be run. Prove provides a useful summary of the results of running many test scripts. The test script output (TAP - test anything protocol) is human-readable, but more easily summarised by a program. That's what prove is. (The volume of test data can be large, a summary is generally more useful unless you're hunting specific failures).

    Are tests embedded into production modules? If not, how does one test (verify) that the act of physically removing tests from production code didn't accidentally interject some errors?

    There are modules for embedding tests, e.g. Test::Inline, but normally the production code is left alone. The production code is normally included into the the .t files via use, so the .t file can call exactly the same code which would run in production.

    I understand that the use of t/ directories to contain .t files is a convention for normal module installation. My problem is that I am coming from the Win32 environment using the ActiveState installation and I therefore rely on ppm for my installs. I don't get to do the "make test" part of the installation so perhaps I just don't have a full appreciation of what testing does. But I do want to have that understanding as I am working on some code I would like to distribute. I am sure development will go better if I can embed testing in it.

    Automated testing is a important and very useful to check you don't break existing code or re-introduce old bugs. Some people go overboard with it, always beware being over-zealous about a methodology or programming practice. But my opinion is that it is very useful, if for no other reason than when you're writing a module (and hence designing an API) it gives you experience in writing code which uses that API - which often leads to a rethinking and refinement process.

    'make test' pretty much just runs prove, so you're not missing out on too much there.

Re: Testing - General Questions
by perlfan (Curate) on Nov 21, 2007 at 15:16 UTC
    As jbert said, tests are in ./t and have the .t extension because they are infact a suite of regression tests that test new features and check for the introduction of new bugs.

    Of the modules that I play around with, I go a step further and have an actual ./tinderbox directory. These scripts are for my own use, and are .pls. I also use lib '../lib';, so that I can test during development with out have to make test

    The purpose of these files is to test for new/old bugs, /and/ to stress the application. For example, my tinderbox tests take days to run versus under 30 seconds for what is in ./t. My *.t tests, however, are subsets of these tinderbox tests, and excercise parts of the code and functionality that I have deemed important.

    You can never test too much, and I for one am greatful that there is a convention, and I am more than happy to follow it.

    Regarding AS Perl, as far as I know perl -MCPAN -e shell still works under it. I suggest you build (and test) your module(s) using the CPAN convention. I can't imagine that it is too difficult to repackages for ppm, and I find it even more unlikely that there is no facilitiy for automated testing....of course, I could be wrong.

    update: blazar properly pointed out that I meant use base when all I wanted was use lib.
      I'd never heard of this notion of having a 'tinderbox' directory before, or the concept behind it - having a superset of tests from which the quick / essential ones are drawn (into .t)

      That's a great idea! I'd always wondered where to store things that were too time consuming to put in tests but too re-usable to just throw away. Thank you for explaining your system.

      where did the word 'tinderbox' come from?

        Glad it could help. I didn't invent it. I see it used in projects like FreeBSD and, of course, Parrot. It is just a bunch of regression tests that I run whenever I make a non-trivial change in the code - or when I am bored and just like to see them run :)

        All I did was create a directory to store them all.
Re: Testing - General Questions
by chromatic (Archbishop) on Nov 21, 2007 at 19:13 UTC
    Are the .t files really the same as the .pl files but with embedded tests?

    You can write your tests in any language you want. Parrot's .t files use quite a mixture of languages.

      I think some versions of Test::Harness assume perl, they invoke "perl foo.t" rather than "foo.t".

      Your point stands though, the assumption is that a .t file will generate something like TAP. How it does it shouldn't matter.

        I think some versions of Test::Harness assume perl, they invoke "perl foo.t" rather than "foo.t".

        Some do, but we file bugs when we notice that, and the latest versions of Test::Harness and TAP::Parser work just fine with Parrot, even on .t files that don't start with a shebang.

Re: Testing - General Questions
by TGI (Vicar) on Nov 21, 2007 at 21:48 UTC

    Update: I need to clarify that I also use this process for internally developed modules, and that the testing phase is absolutely key. Every time I have skimped on testing, I have regretted it.

    Many modules don't have current PPMs available. If I should desire to install one of these modules, I follow these steps:

    1. Download distribution from CPAN.
    2. Unpack distribution to a build directory.
    3. Run Makefile.PL or ?? as appropriate.
    4. Run make
    5. Run make test
    6. Run make_ppm from PPM::Make
    7. Run ppm install Foo.ppd

    I like to stick with one package manager, so I avoid doing manual installs or using the CPAN shell. YMMV.

    The last two steps could be condensed into simply running 'ppm_install'.


    TGI says moo

Re: Testing - General Questions
by TomKane (Beadle) on Nov 21, 2007 at 22:21 UTC
    I want to thank everyone for their replies. I think I've finally gotten it.

    My last experience with QA was in an old fashioned mainframe coder sweatshop. Tests were always performed on the finished apps. The idea of creating tests for the support modules (and we sure had a lot of them) wasn't ever thought of.

    The focus of Perl testing (for developers, at least) is on the functionality of the modules as assessed through their respective interfaces. The finished production apps are taken care of, in due course, if the code is truly modular.

    The indentation on my forehead is now a quarter-inch deeper from slapping it when I saw how obvious the answer is. Now I can continue reading the book "Perl Testing: adn" without the disconnect I was suffering. Thanks again.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://652125]
Approved by marto
Front-paged by friedo
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (5)
As of 2014-12-22 00:03 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (109 votes), past polls