Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Quiet test runner

by adrianh (Chancellor)
on Jul 30, 2005 at 14:39 UTC ( #479609=sourcecode: print w/ replies, xml ) Need Help??

Category: testing
Author/Contact Info Adrian Howard
Description:

Quick and dirty test runner for perl that only reports test failures and unexpectedly succeeding todo tests.

Update: typo fixed, thanks tlm.

Update: fixed is_expected_test_result, thanks tomhukins

#!/usr/bin/perl

use strict;
use warnings;
use Test::Harness::Straps;
use IPC::Run3;

my $strap = Test::Harness::Straps->new();

show_failures( $_ ) for @ARGV;

sub show_failures {
    my $file = shift;
    my $result = test_result( $file );
    show_details( $result ) unless is_expected( $result );
}

sub test_result {
    my $file = shift;
    return { 
        file => $file,
        $strap->analyze( $file, collect_stderr_and_stdout( $file ) )
    };
}

sub is_expected {
    my $result = shift;
    return $result->{ passing } && $result->{ bonus } == 0;
};

sub show_details {
    my $result = shift;
    print "$result->{file}\n", 
        map { "* $_->{name}\n" . diagnostics( $_ ) } 
            grep { ! is_expected_test_result( $_ ) } @{ $result->{ det
+ails } };
}

sub is_expected_test_result {
    my $test_result = shift;
    return $test_result->{ok} &&
        ! is_unexpected_todo_success( $test_result ) &&
        ! diagnostics( $_ );
}

sub diagnostics {
    my $test_result = shift;
    return $test_result->{ diagnostics } if $test_result->{ diagnostic
+s };
    return "    unexpected success from TODO test\n"
        if is_unexpected_todo_success( $test_result );
    return '';
}

sub is_unexpected_todo_success {
    my $test_result = shift;
    return $test_result->{type} eq "todo" && $test_result->{actual_ok}
}

sub collect_stderr_and_stdout {
    my $file = shift;
    my $command = [ $strap->_command, $strap->_switches( $file ) || ()
+, $file ];
    my $output = [];
    run3( $command, \undef, $output, $output ) || die "run3: $!\n";
    return $output;
}

Comment on Quiet test runner
Download Code
Re: Quiet test runner
by tomhukins (Curate) on Sep 21, 2005 at 15:22 UTC

    I find this really useful, thanks. I'm using it as a consequence of the sitation I described in Reporting test failures within cron.

    I noticed it doesn't work quite right if I run an incorrect number of tests. For example:

    use Test::More tests => 3; ok 1, 'Happy'; ok 1, 'Cheerful';

    We expect three tests but only run two, which both succeed. Fortunately, is_expected deals with this, but unfortunately show_details doesn't report the failure. The altered is_expected_test_result below fixes this reporting problem:

    sub is_expected_test_result { my $test_result = shift; return $test_result->{ok} && ! is_unexpected_todo_success( $test_result ) && ! diagnostics( $_ ); }

    I discovered this when running some Test::WWW::Mechanize tests against a server that stops responding, causing later tests not to run.

      Well spotted. Thanks.

Back to Code Catacombs

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: sourcecode [id://479609]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (9)
As of 2015-07-04 10:11 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (59 votes), past polls