Re: Autotesting dependencies compatibility

[MidLifeXis]

The thing that comes to mind immediately is that this would create an exponential explosion in testing complexity. I am not saying that the end goal is not laudable, but that the number of tests that would need to be run is the product of (well, in round numbers):

• the number of versions for each dependent module,
• the number of versions for each child dependency for your dependent modules,
• and so on throughout the dependency graph.

The best compromise I have come up with (by no means assume that I should be an authority on this) is to test at the latest, and include exceptions in your Build.PL, Makefile.PL, or META files. If you require specifying a full dependency chain, then test with a specific set of dependencies, and use the CPAN toolset to lock that configuration down in your environment.

[vsespb]

I would exclude "the number of versions for each child dependency for your dependent modules". (2+ level dependencies).

If XYZ depends on ABC, and ABC depends on DEF.

XYZ should be tested with all versions of ABC.

and ABC should be tested with all versions of DEF.

No need to test XYZ with all versions of DEF.

If XYZ uses DEF directly, it should list it in prereqs. (tools like CPANTS can detect this)

[MidLifeXis]

I disagree. What if your use of ABC passes a parameter to it that tickles an incompatible behavior in a particular version of DEF? Until a test is included in ABC that replicates your usage, the problem will not be uncovered in your testing scenario.

[vsespb]

What if your use of ABC passes a parameter to it that tickles an incompatible behavior in a particular version of DEF?

Then we hope that ABC is a good module and has a testsuite that tests that DEF works well with all parameters, also we hope that ABC validates input.

If this parameter is an complex object of class PQR, that is possible, that DEF requires/uses that class. So all PQR versions tested with DEF. Or maybe XYZ tested with all versions of PQR.

It's unlikely that both use PQR but none "use"'s it.

On the other side it's possible that a module does not have testsuite (except one "use_ok") at all and works like "garbage in - garbage out" or sometimes just "garbage out". Obviously tests won't help there.

Anyway, better test something than nothing.