Controlled CPAN distribution breakage

[dwalin]

I have a set of modules published on CPAN, and I need to push a new major release that will introduce some (controlled) breakage. The problem is somewhat complicated by the fact that the modules in question have interconnected dependencies on each other:

• Foo is the core dependency
• Foo::Bar, Foo::Baz, and Foo::Qux depend on Foo
• Foo::Bar::Fred depends on Foo::Bar
• Foo::Bar::Clunk depends on Foo::Bar::Fred
• Test::Foo depends on Foo::Bar::Fred, and Foo::Bar::Clunk

Practically this means that when core Foo introduces breaking changes, the whole suite must be lifted from version X to version Y. I would like to minimize the impact; a logical way would be to keep both old and new versions independent but I don't see a way to do that with CPAN, except assigning the new version a something_blah number and thus prohibit the automatic tools from trying to upgrade to this "dev" version.

What is a good strategy to deal with this kind of situations? I'm sure it will happen again, if not with this pack of modules then with another, and the next time I'd like to plan ahead.

Regards,
Alex.

[basiliscos]

May be you should take into account the Mojolicious approach: a set of tightly coupled modules are shipped together inside single distribution.

If you draw dependencies graph, and if there is circular references it it, you should think about single distribution, or you should think about breaking that references and release new modules without circular references.

[MidLifeXis]

Is there any way to keep the old API at the same time as adding the new API? You could then add a deprecation warning to the old API methods, so that users of your current API would be aware of the pending changes. At this time, you could increment the MINOR version number (the Y in X.Y), indicating a backward compatible API change. You may want to push the deprecation warning off to the MAJOR version change. This decision depends on if you consider emitting warnings as breaking your API -- I can see arguments either way.

Once that has been available for a sufficient amount of time, then break your old API, perhaps even dying on the old API calls, and increase the MAJOR version number, indicating a backward-incompatible API change.

• X.Y.Z - current version number
• X.Y+1.0 - Add NEW api, leaving old API as is.
• X.Y+2.0 or X+1.0.0 - Add deprecation warning
• X+1.0.0 or X+1.1.0 - Remove old API.

[dwalin]

In fact the API stays mostly compatible, with several changes that should not, in theory, be breaking but I have neither time nor desire to do the proper regression testing so I assume that some breakage will ensue.

I'm more concerned about CPAN tools not making any distinction between major versions. It would be so much more convenient if the auto tools like cpanp and cpanm would take module versions into consideration and didn't jump from 1.x to 2.0 for example, instead following the current 1.x branch until the user makes a conscientious decision to upgrade. Unfortunately that seems not to be the case, and when I release the next major version all the tools will happily adopt it as the newest-and-greatest, thus causing potentially unwanted breakage and grief for the unsuspecting users.

One option is to change the name of the distribution, thus avoiding the problem altogether. That seems silly because in fact it's the same thing; I just want the users to review and retest their code, adjust it to the new version and deploy it as they seem fit instead of forcing them to upgrade at what is usually a very inconvenient time.




Regards,
Alex.

[Anonymous Monk]

It would be so much more convenient if the auto tools like cpanp and cpanm would take module versions into consideration and didn't jump from 1.x to 2.0 for example, instead following the current 1.x branch until the user makes a conscientious decision to upgrade.

Well, hah :)

See https://metacpan.org/pod/CPAN::Meta::Spec#prereqs for the bare-bone-ness

See Task for advanced bones

All you need, to get cpan/cpanm not to jump from 1.x to 2.x is add checks in all the related Makefile.PL equivalents, that does  die "ABORTING; WILL NOT FORCE UPGRADE TO 2X VERSION WITHOUT Task::Foo2X\n" if not require Task::Foo2x;

Then Task::Foo2x does explicitly install  DWALIN/Foo-2.0.1.tar.gz and all the stuff

Then you get everything you want, but naturally you have to update the 1.x version to have this enhanced Makefile.PL equivalents

[Anonymous Monk]

You're not breaking CPAN but your modules on CPAN :)

Also, there is no "Foo" module on CPAN :)

But you have two choices -- break it, and let the chips fall where they may -- or maintain API compatibility (  HTML::Parser->new( api_version => 3, ... )

So, you've decided to change things, great, do whatever you feel will cause the least pain for those (if any) users of your code

a logical way would be to keep both old and new versions independent but I don't see a way to do that with CPAN

Sure you do, make a new namespace, as in Foo2

It basically comes down to promises you made :) If you warned users that API may change in future versions, then they were warned :)

Piece of advice, if your docs/tutorials/videos... are outdated with respect to the current working API, go with the rename ... invalidating existing docs/tutorials with API changes is super weak self inflicted stupidity regardless of the promises you made ... if you can't delete the videos rename :)

See also Update module and break api, or change namespace

I think whatever your product, rename is the simplest, think fork

[dwalin]

Thanks for the tip, I've updated the node title. Hard to think straight at 11pm. :)

Changing the namespace is the last resort. I already have 8 distributions in the old namespace, with 9th coming along and possibly 10th shortly; renaming all that and maintaining the whole mess afterwards looks like a full time job. I kind of hoped to find a relatively easy way to avoid all that...




Regards,
Alex.

[mlawren]

Something I have been considering for a similar situation is the use of the Alt namespace.

What you could do is release Alt::Foo::v1, and also new Alt::Foo::Bar::v1 etc that depends on Alt::Foo::v1. Downstream Makefile.PLs that need the old compatability depend on the Alt* version, but nothing need change in actual code.

This one-time change by your downstream users is perhaps a middle ground