Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot

Quiet test runner

by adrianh (Chancellor)
on Jul 30, 2005 at 14:39 UTC ( #479609=sourcecode: print w/replies, xml ) Need Help??
Category: testing
Author/Contact Info Adrian Howard

Quick and dirty test runner for perl that only reports test failures and unexpectedly succeeding todo tests.

Update: typo fixed, thanks tlm.

Update: fixed is_expected_test_result, thanks tomhukins


use strict;
use warnings;
use Test::Harness::Straps;
use IPC::Run3;

my $strap = Test::Harness::Straps->new();

show_failures( $_ ) for @ARGV;

sub show_failures {
    my $file = shift;
    my $result = test_result( $file );
    show_details( $result ) unless is_expected( $result );

sub test_result {
    my $file = shift;
    return { 
        file => $file,
        $strap->analyze( $file, collect_stderr_and_stdout( $file ) )

sub is_expected {
    my $result = shift;
    return $result->{ passing } && $result->{ bonus } == 0;

sub show_details {
    my $result = shift;
    print "$result->{file}\n", 
        map { "* $_->{name}\n" . diagnostics( $_ ) } 
            grep { ! is_expected_test_result( $_ ) } @{ $result->{ det
+ails } };

sub is_expected_test_result {
    my $test_result = shift;
    return $test_result->{ok} &&
        ! is_unexpected_todo_success( $test_result ) &&
        ! diagnostics( $_ );

sub diagnostics {
    my $test_result = shift;
    return $test_result->{ diagnostics } if $test_result->{ diagnostic
+s };
    return "    unexpected success from TODO test\n"
        if is_unexpected_todo_success( $test_result );
    return '';

sub is_unexpected_todo_success {
    my $test_result = shift;
    return $test_result->{type} eq "todo" && $test_result->{actual_ok}

sub collect_stderr_and_stdout {
    my $file = shift;
    my $command = [ $strap->_command, $strap->_switches( $file ) || ()
+, $file ];
    my $output = [];
    run3( $command, \undef, $output, $output ) || die "run3: $!\n";
    return $output;
Replies are listed 'Best First'.
Re: Quiet test runner
by tomhukins (Curate) on Sep 21, 2005 at 15:22 UTC

    I find this really useful, thanks. I'm using it as a consequence of the sitation I described in Reporting test failures within cron.

    I noticed it doesn't work quite right if I run an incorrect number of tests. For example:

    use Test::More tests => 3; ok 1, 'Happy'; ok 1, 'Cheerful';

    We expect three tests but only run two, which both succeed. Fortunately, is_expected deals with this, but unfortunately show_details doesn't report the failure. The altered is_expected_test_result below fixes this reporting problem:

    sub is_expected_test_result { my $test_result = shift; return $test_result->{ok} && ! is_unexpected_todo_success( $test_result ) && ! diagnostics( $_ ); }

    I discovered this when running some Test::WWW::Mechanize tests against a server that stops responding, causing later tests not to run.

      Well spotted. Thanks.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: sourcecode [id://479609]
[james28909]: yep, i am scraping a websites date/time stamp from an article. none of that is relevant though because the strings i get are as expected. the returned data from the website is the list of @weekdays
[james28909]: i need to convert those @weekdays to an numerical date based off of scalar localtime
[james28909]: i have thught about this and though about it lol
[james28909]: also remember, that if 'Today' is may 1, then Yesterday is not may 31 lol
[james28909]: theoretically speaking ofcourse
[james28909]: so if you subtracting as you loop, be aware that you will eventually be subtracting into a past month, or year <.<
[james28909]: dontcha love it!

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (4)
As of 2017-04-29 03:50 GMT
Find Nodes?
    Voting Booth?
    I'm a fool:

    Results (531 votes). Check out past polls.