Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Bailing out when testing

by sedusedan (Monk)
on Oct 25, 2012 at 02:14 UTC ( #1000748=perlquestion: print w/ replies, xml ) Need Help??
sedusedan has asked for the wisdom of the Perl Monks concerning the following question:

Is there an easy way (I'm thinking a command-line switch in prove) to bail out after the first failure in a test script? Should there be such command-line switch? Sometimes I just want to check the details of the first failure instead of being deluged by tens or hundreds of failures (and wait until the test finishes) when something goes wrong.

Comment on Bailing out when testing
Re: Bailing out when testing
by Anonymous Monk on Oct 25, 2012 at 02:39 UTC

    ?? prove --harness JumpShipAtFirstSignOfTrouble ...

    ?? prove --harness EarlyBail ...

Re: Bailing out when testing
by chrestomanci (Priest) on Oct 25, 2012 at 09:42 UTC

    See the docs for prove.

    There is no documented option to stop on the first faliure, so not directly.

    A couple of suggestions:

    • You could use the --trap command line option to prove, which will cause it to trap ^C and stop the test, With that you can monitor the test run an hit ^C as soon as you see something fail.
    • If you are able to change the source code of the tests, then Test::Most contains a bail_on_fail directive. You can put that at the top of your test script, and it will cause the test script to bail after the first faliure.

    Also how long do your tests take to run, and how many tests are your putting in each .t file? I think you might be doing stuff wrong if each .t file takes more than a few seconds to run. Perhaps you need to break up your tests into many smaller test files, and simplfy what they do to remove longer running operations.

    When I develop tests scripts, I usualy try to limit the number of tests to arround 100 per .t file, so that each individual test file runs quickly. If there are tests that take a significant lenght of time (eg: due to network or database access), I use an environment varable or command line switch to skip them untill all the other tests are working.

    You can also speed up testing via prove by running several tests in parallel. On my (4 core) machine, I usualy run something like:

    find t -name *.t | xargs prove -j9 -s

    find will find all the test file in the test subtree of my source tree and pass them to prove, which will run them in random order, with up to 9 tests running at once. Running multiple tests concurrently is also a good way of finding and removing unexpected dependencies between tests.

      I'm talking about a single test file, so -j is irrelevant here.

      Actually it's not how long the test file runs (which is not more than a few seconds), but the amount of diagnostic output that the failures spew. In my particular case I want just the "head" of the diagnostics.

      Anyway, thanks for the suggestions.

        If you only want to do this occasionally, why not pipe the output to 'head'?

        for(split(" ","tsuJ rehtonA lreP rekcaH")){print reverse . " "}print "\b.\n";
Re: Bailing out when testing
by McDarren (Abbot) on Oct 25, 2012 at 12:35 UTC
      Thanks for the useful links. I'll be considering using Test::Most for testing my (new) distros.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1000748]
Approved by davido
Front-paged by tye
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (5)
As of 2014-12-29 04:39 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (184 votes), past polls