Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris
 
PerlMonks  

RFC: Test::Refute - extensible unified assertion & testing tool

by Dallaylaen (Friar)
on Jan 05, 2017 at 08:28 UTC ( #1178993=perlmeditation: print w/replies, xml ) Need Help??

I would like to present a new assertion/testing tool project. Now there's excellent Test::More with a whole ecosystem built around it and a dozen assertion tools (typically optimizing themselves out in prod environment) available on CPAN. So WHY?

RATIONALE

1. It's more or less compatible with Test::More. Actually it aims to be 100% compatible condition-wise and somewhat compatible regarding the test flow, sans black magic like TODO: and SKIP:. (Saner substitutes planned).

2. It has both functional AND object-oriented ("contract") interface. Contracts can be used in production code, say to check user input or plugin module meet requirements. Nesting contracts is possible (that's how subtest is implemented).

3. It's very easy to extend (see below). A Builder is still required, but all it does is wrapping a given condition around for export AND OO-based usage. The internal check logic does NOT depend on it. Writing a custom Contract backend (e.g. outputting XML, or dying after a certain error count) is also possible.

4. Built with testability in mind. Custom tests' failure modes can be easily checked.

5. It's faster. Up to 5x-7x in a synthetic test (see t/speed.t) and up to 30-50% in real life test suite.

I'd rather continue dreaming about the contract feature, if this rant wasn't the last drop.

SYNOPSIS

The test script usage is quite simple (read: ripped off Test::More):

use Test::Refute; is 42, 42, "this holds"; like "foo", qr/bar/, "this fails"; done_testing; # this is mandatory, even if plan declared

Or the object-oriented interface, runs just happily inside production code (on "oks" on STDOUT, no influence on return code etc):

use Test::Refute::Contract; my $c = Test::Refute::Contract->new; $c->is($user_input, 42); $c->like($another_input, qr/f?o?r?m?a?t/); if ($c->is_valid) ...

Or combining best of both worlds:

use Test::Refute qw(no_init); my $c = contract { is $user_input, 42; like $another_input, qr/f?o?r?m?a?t/; }; # analyze $c here ...

SOME PHILOSOPHY

The whole module is built around a somewhat counter-intuitive, but very simple and powerful interface.

$contract->refute( $condition, $explanation );

can be thought of as an assertion turned inside-out. If $condition is false (or null, if we're talking about Java/C++), it is assumed a pass. Everything is fine, no further details needed.

However, if it is true/present, it means that something is wrong, and the condition itself is considered the reason for failure.

This is similar to a program returning zero on successful completion, but different error codes on failure. Or this is like falsifiability concept in modern science. Or, quoting Leo Tolstoy, "All happy families are alike; each unhappy family is unhappy in its own way".

EXTENDING

Given the above, ALL your test procedure should care about is trying hard to find a discrepancy in its input (aka pure function).

Now since we need to keep OO/functional interface parity, there IS a Moose-ish builder module that works as follows:

build_refute my_is => sub { $_[0] ne $_[1] && "$_[0] != $_[1]"; }, args => 2, export => 1; # take 2 to 3 args; export by default

And this is it! A real is() is a tad harder for under handling & output escaping though. More examples can be found in Test::Refute::Basic.

More primitives (talking to the current contract, init/destroy temp stuff etc) are to follow.

Testing your shiny new test is also trivial, no hacks, forks & pipes required.

use Test::Refute; use My::Test::Module; my $c = contract { my_test (...pass); my_test (...fail); # .... }; $c->sign("100101"); note $c->get_tap; done_testing;

CONCLUSION

The project is in deep alpha stage right now, but I hope it evolves into something usable. Looking forward to your support, critique, suggestions, and prior art links.

https://github.com/dallaylaen/perl-test-refute

Replies are listed 'Best First'.
Re: RFC: Test::Refute - extensible unified assertion & testing tool
by choroba (Bishop) on Jan 05, 2017 at 12:54 UTC
    Have you checked the new Test::Builder? If not, do you support or plan to support subtests, testing forks and threads, and many other hard tasks it provides or makes possible?

    ($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord }map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,

      Thanks for your answer!

      Of course I'm looking at the features, just trying not to byte more than I can chew. As for the mentioned points:

      • subtest support already there, although a bit clumsy.
      • fork'ed subtest/contract planned in the near future.
      • threads - I have no real experience with threads, so I'd rather focus on AnyEvent/Coro who are also no piece of cake to test against.

      I wonder if I can roll out a Builder-based backend as well...

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://1178993]
Approved by Athanasius
help
Chatterbox?
[Corion]: Hmm - I think I'm not really making friends at $work. "We" (as in $current_project) should enter some Changes into the system, and I just told the Transition Manager, who should do that or tell somebody to do that to learn the system in which they ...
[Corion]: ... should create that Change.
[Corion]: The nicer approach maybe would have been to indirectly tell them that, or tell them who could teach them...
[erix]: No More Mr Nice Guy !

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (10)
As of 2017-10-23 12:12 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My fridge is mostly full of:

















    Results (279 votes). Check out past polls.

    Notices?