redirecting output from a format

[mpeters]

I am trying to redirect the output from the Test::Harness::runtests() method into a scalar. I'm using IO::Scalar to do the job of redirecting STDOUT and it works just like expected, except for one spot. When a subtest fails, Test::Harness uses a format (associated with STDOUT) to create the pretty report that shows which subtests failed, etc. I would also like to capture this report into my scalar, but I haven't been able to do it.

    my $test_output;
    my $IO = IO::Scalar->new(\$test_output);
    $IO->format_name('STDOUT');
    $IO->format_top_name('STDOUT_TOP');
    select($IO);
    eval {
        runtests(@testfiles);
    };
    $test_output .= $@ if ($@);
    select(STDOUT);

After doing this, my scalar ($test_output) does not contain the failure report and neither is it printed to the screen. When I try this:

    my $test_output;
    tie(*STDOUT, 'IO::Scalar', \$test_output);
    eval {
        runtests(@testfiles);
    };
    $test_output .= $@ if ($@);
    untie(*STDOUT);

I end up with the same data in $test_output, but the failure report is printed to the screen. Still not what I want... Any suggestions?

[brian_d_foy]

I ran into a similar problem and came up with a very ugly, don't-try-this-at-home solution I posted in use.perl

 my( $totals, $failed ) = eval {
            local @INC = @INC;
            local *STDOUT;                # Shut up Test::Harness!
            local *STDERR;                # Yeah, you too!
            my $MM = ExtUtils::MM->new(); # and you!
 
            unshift @INC, map { File::Spec->rel2abs($_) }
                @{ $MM }{ qw( INST_LIB INST_ARCHLIB ) };
 
            Test::Harness::_run_all_tests( @test_files )
            };

I could use Test::Harness::Straps to do this all myself, but at the time I was on a bit of a deadline and wasn't sure I could reproduce everything _run_all_tests did, so I just wrapped it.

This solution is a slightly (very slightly) easier than shelling out to `make test` and parsing the output since I only needed the percentage of tests that fail. Besides, I was curious if it would work.

To repeat: very ugly kludge for demonstration purposes only. If you truly, deeply, and sincerely need to do this sort of thing, look at Test::Harness::Straps which can run everything for you and hand you back the information. I haven't done that yet because I've been little-l lazy (or big b Busy).

[Flame]

Sorry, it's a well known fact that you can't use IO::String to redirect format calls. format and IO::String As you can see there, I've tried it in the past.

[ysth]

You can't use IO::Scalar, but you can use PerlIO::scalar in 5.8.x:

format FOO=
@||| @||| @|||
$x,$y,$z
.
$x=1;$y=2;$z=3;
$save = "";
open FOO, ">", \$save;
write FOO;
print "got:$save!\n";

Reopening STDOUT is a little more complicated, but easily doable.

[jZed]

You need to tie STDERR to a scalar also.

[mpeters]

I did try that in my initial experimentation, and I revisited it again after your suggestion, but still no dice... after I redirected STDERR to a scalar I ended up with nothing there. I did this:

    my $test_output;
    my $test_err;
    tie(*STDOUT, 'IO::Scalar', \$test_output);
    tie(*STDERR, 'IO::Scalar', \$test_err);
    eval {
        runtests(@testfiles);
    };
    $test_output .= $@ if ($@);
    untie(*STDOUT);
    untie(*STDERR);
    $test_output .= $test_err;

$test_output was no different than it was before. No failed test report.

[petdance]

Why do you want the output from Test::Harness in a scalar? What do you plan to do with it?

[mpeters]

I'm trying to capture it and send it in an email... automated build with an accompanying test whose results are sent to the development team

[petdance]

So why bother capturing runtests()? Catch the output of make test or prove.

Here's the smokebot that we use. At some point I should put it into Test::Harness proper.

#!/bin/sh

if [ $# -lt 2 ]
then
    echo Must pass at least a branch, and one email address,.
    echo plus any parms to echo pass to smoke.
    exit 1
fi

BRANCH=$1
shift

MAIL=$1
shift

cd $TMP
DIR=tw
FULLPATH=$TMP/$DIR

# This assumes you have already logged in once as anoncvs
# so that the password is in your ~/.cvspass file.
#cvs -d/home/cvs -Q co -d $DIR -r $REV tw > /dev/null
svn co file:///home/svn/tw/$BRANCH $DIR > /dev/null

TWROOT=$FULLPATH
export TWROOT

/home/smoke/tw/Dev/devapache stop  >  /dev/null 2>&1
/home/smoke/tw/Dev/devapache start > /home/smoke/smoke.out 2>&1.

cd $TWROOT
smoke $@ >> /home/smoke/smoke.out 2>&1.
grep -i "^Failed" /home/smoke/smoke.out > /home/smoke/smoke.out.fail

if [ -s /home/smoke/smoke.out.fail ]
then
    STATUS="FAILED"
    mail -s"Smoke $REV $@ $STATUS `date`" $MAIL < /home/smoke/smoke.ou
+t
else
    STATUS="passed"
fi

/home/smoke/tw/Dev/devapache stop >> /home/smoke/smoke.out 2>&1.

Note that smoke is an early predecessor to prove, so modify for your tastes.