Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?

Will I break CPAN?

by Tux (Abbot)
on Mar 03, 2014 at 14:46 UTC ( #1076783=CUFP: print w/replies, xml ) Need Help??

Once one of your modules on CPAN is used/required by another module on CPAN not under your own control, it might be a good idea to test all those modules with your new code to verify you didn't break anything.

The first thing you want to check is the fact that your module actually is used by something else. We are in luck here, as that is already done for us. For this example, I'll continue with a short and simple module (Data::Peek) as the logs are much shorter, but the process is exactly the same as for a module that is used widely, as DBI or Text::CSV_XS, for which I wrote this to begin with.

Before you want to check other modules, of course you check your own suite:

$ perl Makefile.PL $ make test : All tests successful. : Result: PASS $ make distcheck

So the module passes, but does it still pass in the modules that require it? To test that, we first need to see what that lists consists of. CPANTS has that list readily available for each module on CPAN. We can use the internals of CPAN to fetch and test that list.

$ cat sandbox/ #!/usr/bin/perl use 5.16.2; use warnings; use LWP; use LWP::UserAgent; use HTML::TreeBuilder; use CPAN; use Capture::Tiny qw( :all ); use Test::More; my $tm = "Data-Peek"; # The module I want to check my %tm = map { $_ => 1 } qw( ); $| = 1; $ENV{AUTOMATED_TESTING} = 1; # Skip all dists that # - are FAIL but not due to Data::Peek # - that require interaction (not dealt with in distroprefs) # - are not proper dists (cannot use CPAN's ->test) # - require external connections or special devices my %skip = map { $_ => 1 } qw( GSM-Gnokii ); my $url = "$tm/used_by"; my $ua = LWP::UserAgent->new (agent => "Opera/12.15"); my $rsp = $ua->request (HTTP::Request->new (GET => $url)); $rsp->is_success or die "get failed: ", $rsp->status_line, "\n" +; my $tree = HTML::TreeBuilder->new; $tree->parse_content ($rsp->content); foreach my $a ($tree->look_down (_tag => "a", href => qr{/dist/})) { (my $h = $a->attr ("href")) =~ s{.*dist/}{}; exists $skip{$h} || $h =~ m{^( $tm (?: $ | / ) | Task- )\b}x and n +ext; (my $m = $h) =~ s/-/::/g; $tm{$m} = 1; } my %rslt; foreach my $m (sort keys %tm) { my $mod = CPAN::Shell->expand ("Module", "/$m/") or next; # diag "$m ... "; $rslt{$m} = [ [], capture { $mod->test } ]; $rslt{$m}[0] = [ $?, $!, $@ ]; is ($?, 0, $m); } done_testing;

That fetches the list of modules that require Data::Peek, check if any of those is recognized by CPAN, for each makes a call to CPAN's test function/method, catches all the output with Capture::Tiny (and ignores all output for now) and use the return code available in $? to decide FAIL or success:

$ prove -vwb sandbox/ sandbox/ .. Reading '/home/merijn/.cpan/Metadata' Database was generated on Mon, 03 Mar 2014 14:17:02 GMT CPAN: YAML loaded ok (v0.90) Reading 3 yaml files from /home/merijn/.cpan/build/ ..........................DONE Restored the state of 1 (in 0.0431 secs) ok 1 - App::tkiv ok 2 - Geo::KML ok 3 - Spreadsheet::Read 1..3 ok All tests successful. Files=1, Tests=3, 13 wallclock secs ( 0.02 usr 0.00 sys + 10.29 cusr + 0.60 csys = 10.91 CPU) Result: PASS $

Running this for Text::CSV_XS resulted in 4 RT tickets with patches on other modules that failed depending on Text::CSV_XS where they did either something wrong or were not yet up to date with new(er) code.

If I find myself not forgetting to run this before I release a new module, I think I will prevent a lot of complaints later :)

Enjoy, Have FUN! H.Merijn

Replies are listed 'Best First'.
Re: Will I break CPAN?
by tobyink (Abbot) on Mar 03, 2014 at 17:02 UTC

    Have you seen Test::DependentModules?

    use Moops; class Cow :rw { has name => (default => 'Ermintrude') }; say Cow->new->name

      Yes, I did, after reading Moose test file I was pointed to, which also uses MetaCPAN::API. By that time my own script already worked.

      Moose does have a good reason to roll its own. Look at the list of expected failures, which they nicely documented.

      I used the modules I already had available and did not realize I also had MetaCPAN::API installed. Now I do.

      I see a value for all three approaches. Maybe I'll simplify to using Test::DependentModules now that I am aware of it. I did look, but probably typoed on Dependant :/

      $ modlist depending $ modlist dependant Module name Description + Version Statistics::DependantTTest $ modlist dependent Module name Description + Version MooseX::Attribute::Dependent MooseX::Attribute::Dependent::Meta::Role::ApplicationToClass MooseX::Attribute::Dependent::Meta::Role::ApplicationToRole MooseX::Attribute::Dependent::Meta::Role::Attribute MooseX::Attribute::Dependent::Meta::Role::Class MooseX::Attribute::Dependent::Meta::Role::Composite MooseX::Attribute::Dependent::Meta::Role::Method::Accessor MooseX::Attribute::Dependent::Meta::Role::Method::Constructor MooseX::Attribute::Dependent::Meta::Role::Role Mpp::BuildCheck::architecture_independent Test::DependentModules WAIT::Table::Independent $

      My English can probably be improved a lot

      Enjoy, Have FUN! H.Merijn
Re: Will I break CPAN?
by etj (Sexton) on Aug 26, 2014 at 09:04 UTC

        It is just sad that that site only returns HTML or XML, and no useful formats like JSON or CSV.

        Enjoy, Have FUN! H.Merijn
        I wasn't saying EUMM isn't used. I was pointing out the site given ( isn't getting reverse dependencies right. Good to see isn't having a similar problem.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: CUFP [id://1076783]
Approved by ww
Front-paged by Arunbear
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (3)
As of 2017-10-21 01:43 GMT
Find Nodes?
    Voting Booth?
    My fridge is mostly full of:

    Results (269 votes). Check out past polls.