RFC: Test::Block

[adrianh]

Update: A somewhat modified Test::Block is now available on CPAN.

---

Ever wanted to specify the number of tests your test script runs at a finer level of granularity than the whole test script? If so here's one possible solution.

Comments, as ever, welcome. Full code with tests can be downloaded at http://www.quietstars.com/perl/Test-Block-0.03.tar.gz.

package Test::Block;
use strict;
use warnings;
use Test::Builder;

our $VERSION = '0.03';

my $Test = Test::Builder->new;

sub expecting {
    my ($class, $value) = @_;
    bless {
        expected_tests => $value, 
        initial_test => $Test->current_test,
    }, $class;
};

sub _ran { $Test->current_test - shift->{initial_test} };

sub remaining { 
    my $self = shift;
    $self->{expected_tests} - _ran($self);
};

sub DESTROY {
    my $self = shift;
    my ($ran, $expected) = (_ran($self), $self->{expected_tests});
    $Test->ok(0, "block expected $expected test(s) and ran $ran")
            unless $ran == $expected;
};

1;
__END__

=head1 NAME

Test::Block - specify fine granuality test plans

=head1 SYNOPSIS

  use Test::More 'no_plan';
  use Test::Block;

  {
      # This block should run exactly two tests
      my $block = Test::Block->expecting(2);
      ok(1);
      ok(1);
  };

  SKIP: {
      # This block should run exactly three tests
      my $block = Test::Block->expecting(3);
      ok(1);
      skip "skip remaining tests in block", $block->remaining;
  };

=head1 DESCRIPTION

This module allows you to specify the number of expected tests at a fi
+ner level of granuality than an entire test script. It is built with 
+L<Test::Builder> and plays happily with L<Test::More> and friends.

If you are not already familiar with L<Test::More> now would be the ti
+me to go take a look.

=over 4

=item B<expecting>

You create a Test::Block object with the C<expecting> class method, sp
+ecifying the number of tests. When the object is destroyed it creates
+ a failed test if the expected number of tests have not run. For exam
+ple doing:

  {
      my $block = Test::Block->expecting(3);
      ok(1);
      # oops - missed two tests out
  }

will produce

  ok 1
  not ok 2 - block expected 3 test(s) and ran 1

=item B<remaining>

You can find out the number of remaining tests in the block by calling
+ the C<remaining> method on the object. This can be useful in C<SKIP>
+ blocks, for example:

  SKIP: {
      my $block = Test::Block->expecting(5);
      my $pig = Pig->new;
      isa_ok($pig, 'Pig')
          || skip "cannot breed pigs", $block->remaining;
      can_ok($pig, 'takeoff')
          || skip "pigs don't fly here", $block->remaining;
      ok($pig->takeoff, 'takeoff')
          || skip "takeoff failed", $block->remaining;
      ok( $pig->altitude > 0, 'Pig is airborne' );
      ok( $pig->airspeed > 0, '  and moving'    );
  };

If you run this test in an environment where "Pig->new" worked and the
+ takeoff method existed, but failed when ran, you would get:

  ok 1 - The object isa Pig
  ok 2 - can takeoff
  not ok 3 - takeoff
  ok 4 # skip takeoff failed
  ok 5 # skip takeoff failed

=back

=head1 BUGS

None known at the time of writing. 

If you find any please let me know by e-mail, or report the problem wi
+th L<http://rt.cpan.org/>.


=head1 TO DO

Nothing at the time of writing.

If you think this module should do something that it doesn't do at the
+ moment please let me know.


=head1 ACKNOWLEGEMENTS

Thanks to chromatic and Michael G Schwern for the excellent Test::Buil
+der, without which this module wouldn't be possible.

Thanks to Michael G Schwern and Tony Bowden for the mails on perl-qa@p
+erl.org that sparked the idea for this module.


=head1 AUTHOR

Adrian Howard <[e-mail]>

If you can spare the time, please drop me a line if you find this modu
+le useful.


=head1 SEE ALSO

L<Test::Builder> provides a consistent backend for building test libra
+ries. The following modules are all built with L<Test::Builder> and w
+ork well together.

=over 4

=item L<Test::Simple> & L<Test::More>

Basic utilities for writing tests.

=item L<Test::Class>

Easily create test classes in an xUnit style. Test::Class allows you t
+o specify the number of tests on a method-by-method basis.

=back


=head1 LICENCE

Copyright 2003 Adrian Howard, All Rights Reserved.

This program is free software; you can redistribute it and/or modify i
+t under the same terms as Perl itself.

=cut

1;

[Anonymous Monk]

So when a developer or somebody wishes to run the full test suite, they have to modify a test file? They can't just do something like `make test TEST_ALL=1'? Sure I can somehow code that into my test scripts, but why should (your module is too short, it needs more features)?

[adrianh]

So when a developer or somebody wishes to run the full test suite, they have to modify a test file? They can't just do something like `make test TEST_ALL=1'?

Nope :-)

I think you're misunderstanding the purpose. It's not restricting the tests run, but allowing you to specify the number of tests at a finer level of detail.

Some examples of where this may be useful:

• When you have a large test suite keeping the number of tests in the plan in sync with the number of tests in the script can become annoying, since the plan is specified at the top of the script. Lots of scrolling up and down. With Test::Block you can keep the number of tests close to the tests. So instead of:

  use Test::More tests => 100;
  
  isa_ok my $o = Foo->new, 'Foo';
  
  lives_ok { $o->foo(42) } 'can set foo';
  is( $o->foo, 42, 'foo set' );
  
  # 95 other tests 
  
  lives_ok { $o->bar(42) } 'can set foo';
  is( $o->bar, 42, 'foo set' );
  

  you can do:

  use Test::More 'no_plan';
  
  {
      my $b = Test::Block->expecting(1);
      isa_ok my $o = Foo->new, 'Foo';
  };
  
  {
      my $b = Test::Block->expecting(2);
      lives_ok { $o->foo(42) } 'can set foo';
      is( $o->foo, 42, 'foo set' );
  }
  
  # 95 other tests
  
  {
      my $b = Test::Block->expecting(2);
      lives_ok { $o->bar(42) } 'can set foo';
      is( $o->bar, 42, 'foo set' );
  }
  

• It can make maintaining SKIP blocks easier. For example if you have:

  SKIP: {
      ok(1);
      skip "skip tests", 2 if $foo;
      ok(2);
      skip "skip tests", 1 if $bar;
      ok(3);
  };
  

  and add another test at the end of the block you have to change both skip calls to take the new test into account. With Test::Block you would only have to alter the number of expected tests:

  SKIP: {
      my $b = Test::Block->expecting(4);
      ok(1);
      skip "skip tests", $b->remaining if $foo;
      ok(2);
      skip "skip tests", $b->remaining if $bar;
      ok(3);
      ok(4);
  };
  

• If part of your test script runs an indeterminate number of tests then you have to make you're entire script 'no_plan' - negating the possible advantages of having a predetermined plan. With Test::Block you can check that the fixed tests actually run as expected.

• With some test scripts you can only determine the number of tests at runtime. This can lead to complex plans like:

  plan tests => @foo + keys %bar + (3*$ni);
  

  where the plan is a long way from the tests. With test block you can localise the plans with the tests.

  {
      my $b = Test::Block->expecting(scalar(@foo));
      # @foo tests
  }
  {
      my $b = Test::Block->expecting(scalar(%bar));
      # %bar tests
  }
  {
      my $b = Test::Block->expecting(3 * $ni);
      # $ni tests
  }
  

Sure I can somehow code that into my test scripts, but why should (your module is too short, it needs more features)?

What else should it do? Personally I don't think that "too short" is a criticism ;-)

[jplindstrom]

Your examples makes excellent sense along the lines of

http://c2.com/cgi/wiki?LocalityOfReferenceDocumentation

The principle may be stated formally as if it were a Newtonian Law of sorts:

The likelihood of keeping all or part of a software artifact consistent with any corresponding text that describes it, is inversely proportional to the square of the cognitive distance between them.

A less verbose, less pompous description would be simply: Out of sight; out of mind!

Therefore, it is desirable for us to try and minimize the cognitive distance between artifacts and their descriptions!

/J