[solved] Test module to compare results between different runs (save/load)

[Dallaylaen]

Hello dear esteemed monks!

Update: Test::Regression

Is there already a module that allows to run a piece of code and, depending on arguments, either write results to a file, or compare with a previous such file? And is there a name for the concept in general? Searching for save load test on CPAN was somewhat cluttered...

So I need something along the lines of:

use Test::More;
use Test::SaveLoad;
use My::Module; # system under test

# load previous results & compare
my $sl = Test::SaveLoad->new( load => $filename );

# save results for future (in Dumper?..)
my $sl = Test::SaveLoad->new( save => $filename );

# do actual testing
plan tests => 1 if $sl->is_testing;

$sl->test( "Basic usecase" => sub {
    My::Module->load_data( id => 42 );
});
# returns eq_or_diff-like line-by-line comparison

It's not that hard to write, but maybe there is something like that?

thank you

[choroba]

This process is well known under the name "Regression Testing". See Test::Regression for a Perl implementation.

[Dallaylaen]

Thanks!

(Although I thought regression testing is a more general concept, just "find out if we broke something")

[AnomalousMonk]

... I thought regression testing is a more general concept, just "find out if we broke something" ...

It is (see Regression testing). (I like the word "just" in that quote.) In the most common notion of regression testing, you attempt to define all critical inputs and their correct outputs, a test set that can grow very large. (Can we call this "absolute" regression testing?)

In the process you describe in the OP, you "just" take some input data set, run it through a process, and whatever the heck the output is, that's the correct output; if the next run has a different output, something broke. (Perhaps we can call this "differential" regression testing.)

[1nickt]

Hi, if you are using a newish Perl you can apply the --state flag to prove, with options governing which tests to rerun and compare with the previous run's state.

See https://perldoc.perl.org/5.26.0/prove.html#--state for more.

Hope this helps!

[Dallaylaen]

That's not what I was asking for, but I definitely need to check it out. Looks quite useful. Thanks!