Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Quiet test runner

by adrianh (Chancellor)
on Jul 30, 2005 at 14:39 UTC ( #479609=sourcecode: print w/ replies, xml ) Need Help??

Category: testing
Author/Contact Info Adrian Howard
Description:

Quick and dirty test runner for perl that only reports test failures and unexpectedly succeeding todo tests.

Update: typo fixed, thanks tlm.

Update: fixed is_expected_test_result, thanks tomhukins

#!/usr/bin/perl

use strict;
use warnings;
use Test::Harness::Straps;
use IPC::Run3;

my $strap = Test::Harness::Straps->new();

show_failures( $_ ) for @ARGV;

sub show_failures {
    my $file = shift;
    my $result = test_result( $file );
    show_details( $result ) unless is_expected( $result );
}

sub test_result {
    my $file = shift;
    return { 
        file => $file,
        $strap->analyze( $file, collect_stderr_and_stdout( $file ) )
    };
}

sub is_expected {
    my $result = shift;
    return $result->{ passing } && $result->{ bonus } == 0;
};

sub show_details {
    my $result = shift;
    print "$result->{file}\n", 
        map { "* $_->{name}\n" . diagnostics( $_ ) } 
            grep { ! is_expected_test_result( $_ ) } @{ $result->{ det
+ails } };
}

sub is_expected_test_result {
    my $test_result = shift;
    return $test_result->{ok} &&
        ! is_unexpected_todo_success( $test_result ) &&
        ! diagnostics( $_ );
}

sub diagnostics {
    my $test_result = shift;
    return $test_result->{ diagnostics } if $test_result->{ diagnostic
+s };
    return "    unexpected success from TODO test\n"
        if is_unexpected_todo_success( $test_result );
    return '';
}

sub is_unexpected_todo_success {
    my $test_result = shift;
    return $test_result->{type} eq "todo" && $test_result->{actual_ok}
}

sub collect_stderr_and_stdout {
    my $file = shift;
    my $command = [ $strap->_command, $strap->_switches( $file ) || ()
+, $file ];
    my $output = [];
    run3( $command, \undef, $output, $output ) || die "run3: $!\n";
    return $output;
}

Comment on Quiet test runner
Download Code
Replies are listed 'Best First'.
Re: Quiet test runner
by tomhukins (Curate) on Sep 21, 2005 at 15:22 UTC

    I find this really useful, thanks. I'm using it as a consequence of the sitation I described in Reporting test failures within cron.

    I noticed it doesn't work quite right if I run an incorrect number of tests. For example:

    use Test::More tests => 3; ok 1, 'Happy'; ok 1, 'Cheerful';

    We expect three tests but only run two, which both succeed. Fortunately, is_expected deals with this, but unfortunately show_details doesn't report the failure. The altered is_expected_test_result below fixes this reporting problem:

    sub is_expected_test_result { my $test_result = shift; return $test_result->{ok} && ! is_unexpected_todo_success( $test_result ) && ! diagnostics( $_ ); }

    I discovered this when running some Test::WWW::Mechanize tests against a server that stops responding, causing later tests not to run.

      Well spotted. Thanks.

Back to Code Catacombs

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: sourcecode [id://479609]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (4)
As of 2015-09-05 13:04 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My preferred temperature scale is:










    Results (152 votes), past polls