Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?

Verifying your distribution's revdeps still work after a change

by stevieb (Abbot)
on Dec 28, 2017 at 20:48 UTC ( #1206362=perlmeditation: print w/replies, xml ) Need Help??

In one of my pieces of software, Mock::Sub, I recently found that I wanted to add a new feature, and as always, I wrote a test file for the new work before modifying code. After making the first round of changes, I stumbled upon a previously unknown bug, so I opened an issue, and decided to tackle that before adding the new feature, to ensure all previous tests would run.

The new feature is quite minor and actually requires a specific parameter to be sent in to change existing behavour (ie. no existing code that uses the distribution should have been affected), but the bug was a little more complex, and did change things internally.

After I got the bug and new feature added, and before just blindly uploading it to the CPAN, I of course wanted to know whether the reverse dependencies (down-river distributions) would not be adversely affected, which would cascade Testers failure emails to the poor souls who's distributions I broke (most are mine in this case, but I digress).

A long time ago, I went about writing a completely autonomous testing platform to do extensive testing on my repos against all Perlbrew/Berrybrew installations installed (it can dispatch out to remote systems as well). This distribution is Test::BrewBuild One of the core features I built into this software, is to automatically perform unit tests on all reverse dependencies as they currently sit on the CPAN against the changed code in the local distribution.

I'll get right to it; it's pretty straightforward:

First, ensure you're in your repository directory, and ensure all tests pass on the distribution you've just updated:

~/devel/repos/mock-sub$ make test t/00-load.t .................... ok t/01-called.t .................. ok t/02-called_count.t ............ ok t/03-instantiate.t ............. ok t/04-return_value.t ............ ok t/05-side_effect.t ............. ok t/06-reset.t ................... ok t/07-name.t .................... ok t/08-called_with.t ............. ok t/09-void_context.t ............ ok t/10-unmock.t .................. ok t/11-state.t ................... ok t/12-mocked_subs.t ............. ok t/13-mocked_objects.t .......... ok t/14-core_subs.t ............... ok t/15-remock.t .................. ok t/16-non_exist_warn.t .......... ok t/17-no_warnings.t ............. ok t/18-bug_25-retval_override.t .. ok t/19-return_params.t ........... ok t/manifest.t ................... skipped: Author tests not required fo +r installation t/pod-coverage.t ............... skipped: Author tests not required fo +r installation t/pod.t ........................ skipped: Author tests not required fo +r installation All tests successful. Files=23, Tests=243, 1 wallclock secs ( 0.05 usr 0.02 sys + 0.72 cu +sr 0.07 csys = 0.86 CPU) Result: PASS

So far, so good (of course, I had already ensured the "skipped" tests pass as well). Now, after installing Test::BrewBuild, and ensuring you've got at least one instance of Perlbrew/Berrybrew installed, simply run the brewbuild binary, with the -R or --revdep flag. In the example below, for brevity, I've limited the testing against only the version of Perl I'm currently using. If I had tested against more versions (or left off the -o or --on flag it tests against all installed versions by default), each version would be listed under each revdep with the PASS or FAIL status:

~/devel/repos/mock-sub$ brewbuild -R -o 5.24.1 reverse dependencies: App::RPi::EnvUI, RPi::DigiPot::MCP4XXXX, Devel::Examine::Subs, PSGI::Hector, File::Edit::Portable, Devel::Trace +::Subs App::RPi::EnvUI 5.24.1 :: PASS RPi::DigiPot::MCP4XXXX 5.24.1 :: PASS Devel::Examine::Subs 5.24.1 :: PASS PSGI::Hector 5.24.1 :: PASS File::Edit::Portable 5.24.1 :: PASS Devel::Trace::Subs 5.24.1 :: PASS

That's all there is to it. Now I am confident that my changes will absolutely not break any of the down-river distributions that require this one.

Note: If there had of been failures, a bblog directory will be created, and the full test output of that distribution located into its own file for easy review as to what went wrong. The file contains everything related to the test run that you'd normally see output by the cpanm command. Example:

~/devel/repos/mock-sub$ ll bblog drwx------ 2 steve steve 4096 Dec 28 10:53 . drwxrwxr-x 8 steve steve 4096 Dec 28 10:53 .. -rw-rw-r-- 1 steve steve 8128 Dec 28 10:26 App-RPi-EnvUI-5.24.1-FAIL.b +blog

Note 2: If you want to get an understanding of most of the stuff brewbuild is doing, simply throw in a -d 7 to enable full debug logging to stdout.

Update: Here's an example with multiple versions of Perl installed:

App::RPi::EnvUI 5.24.1 :: PASS 5.18.4 :: FAIL 5.24.0 :: FAIL RPi::DigiPot::MCP4XXXX 5.18.4 :: PASS 5.24.0 :: PASS 5.24.1 :: PASS Devel::Examine::Subs 5.18.4 :: PASS 5.24.0 :: PASS 5.24.1 :: PASS PSGI::Hector 5.18.4 :: PASS 5.24.0 :: PASS 5.24.1 :: PASS File::Edit::Portable 5.18.4 :: PASS 5.24.0 :: PASS 5.24.1 :: PASS Devel::Trace::Subs 5.18.4 :: PASS 5.24.0 :: PASS 5.24.1 :: PASS
... and the resulting bblog directory entries:

-rw-rw-r-- 1 steve steve 1085954 Dec 28 13:20 App-RPi-EnvUI-5.18.4-FAI +L.bblog -rw-rw-r-- 1 steve steve 1078097 Dec 28 13:20 App-RPi-EnvUI-5.24.0-FAI +L.bblog

The output in the FAIL logs show that one of my dependencies for the distribution is behind a version on those two versions of Perl, so all I have to do is bump it in the Makefile.PL and re-run the tests. It has nothing to do with the Mock::Sub distribution at all, but did point out a different problem entirely solely with that dist. So I suppose that this tool is handy for warning about other issues outside of the current dist you're updating.

Replies are listed 'Best First'.
Thanks! [JT]
by Dallaylaen (Hermit) on Dec 29, 2017 at 14:58 UTC

    Thank you for the tool.

    It should be linked to on CPAN login page.

      Thank you very much, but I don't know about the link ;)

      My favourite piece is the remote testing capabilities. You can pitch test runs to either Windows/berrybrew and/or Unix/perlbrew installations, as long as you have Test::BrewBuild installed, and a tester running on them. You then go into your repository, and use the bbdispatch application to send out test runs (four run in parallel by default, which is configurable, and there is no limit to the number of remote test boxes to test on). This allows you to literally, with a single command, test against as many platforms and versions of Perl that you've got available:

      Tester box 1 (Windows):

      c:\> bbtester start

      Tester box 2 (Unix) and dispatcher:

      ~/repos/mock-sub $ bbtester start ~/repos/mock-sub $ bbdispatch -t localhost -t - MSWin32-x64-multi-thread 5.26.0_64 :: PASS localhost - x86_64-linux 5.18.4 :: PASS 5.24.0 :: PASS

      Using the -c aka --cmd command to bbdispatch and specifying a specific command string with the brewbuild command, you can also run against your revdeps on all remote testers:

      ~/repos/mock-sub $ bbdispatch -t localhost -t -c 'brewb +uild -R'

      One more tidbit; You can set the system up to monitor a git repository, and automatically run your tests remotely every time the repository is updated. I use this functionality with a Raspberry Pi that has a 4 row by 20 col LCD screen that shows the status of testing, the commit etc, and also emails me on failure.

      # -a, --auto bbtester start -a ... # -a, --auto # -r, --repo (you don't need the piece) bbdispatch -t ... -r stevieb9/mock-sub -a

      Of course, Git must be installed on all tester machines. By default, each tester keeps the fetched repos and test data in the brewbuild directory under your home directory (on both win and nix).

      Happy New Year!


Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://1206362]
Front-paged by Arunbear
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (7)
As of 2019-02-21 18:11 GMT
Find Nodes?
    Voting Booth?
    I use postfix dereferencing ...

    Results (114 votes). Check out past polls.

    • (Sep 10, 2018 at 22:53 UTC) Welcome new users!