Re^2: Maintaining Local CPAN Patches

[Ovid]

Regrettably, it's unlikely that even newer versions of these modules will be used:

• It's a small performance hit (UNIVERSAL::isa alone cost us 20 seconds in our test suite)
• We're finding so many bugs (even segfaults) with code which has a global effect that we try to eliminate all of it as soon as we see it.

It's nothing specific against your modules, but their small impact coupled with the small impact of several other "global scope" modules has created a large impact in terms of performance, bugs, and time spent resolving these issues.

Also, 20 seconds seems like a lot, but our test suite was taking almost 11 minutes to run at the time we removed it and thus 20 seconds was only about 3% of the entire runtime. However, finding just a few modules to remove which gain us 20 seconds each and we can quickly and easy gain a 10% improvement in our test suite (just yesterday I shaved 50 seconds be reimplementing part of Test::Strict internally, but we're up to over 16 minutes again). We have to be aggressive about test performance because it's a huge drain on resources.

[chromatic]

I feel your pain (just look at Parrot's test suite, and then compare it to the Pugs test suite...), but everything I've seen you write publicly about speeding up your test suite has been microoptimizations that gain you five seconds here and ten seconds there.

Now I'm obviously not privy to the details, and you may have 500,000 assertions that run in 11 minutes (which is a pretty good clip), but my instincts tell me to look for the equivalent of algorithmic improvements, not two and three percent here and there.

I don't remember the name of the health insurance billing project you saw me work on, but I did a brief statistical analysis of their test database, dropped 90% of the records, and cut the running time of certain tests by an order of magnitude. I hope you've looked at those types of optimizations and are just doing cleanup now.