Re: Re: Favourite modules March 2004

[Juerd]

Why are a large number of tests a "Bad Thing"(tm)?

They become a bad thing when you have 157732 tests for 2245 lines of code. That is 70 tests per line of code, while of course a lot of lines don't even contain any logic. make test for Regexp::Common takes too long. I've already met a sysadmin who refused to install it because of that and I'm slowly becoming one myself.

if two pieces of code were in front of me and 1 passed 100 unique tests and the other 1000 unique tests, I'd be more confident initially with the latter

Of course, but do you really need thousands of tests for zipcodes? Are these thirty six thousand tests really needed for a palindrome regex?

Plus, don't you install each distribution only once in a while?

Yes, but I dislike this test suite that takes minutes to run for the same reason I dislike compiling from source. I am lazy and impatient and that is exactly why I like Perl.

There's only one thing regarding testing that I dislike more than an exaggerated test suite like Regexp::Common's and that is not having tests at all. I think R::C could do with only, say, a tenth or maybe a hundredth of its current test suite. For development, a huger one might be nice, but 1.5e5 tests on installation is more than I'd like.

[dragonchild]

I'm still confused. What's wrong with kicking off an installation and going to get a cigarette or a cup of coffee? Kick it off, then go to lunch. Or, kick it off at home and go watch the latest SG-1 episode ... That's what I do when I'm installing OSes or Apache, and those take over an hour each. *shrugs* I guess that it seems a little too impatient, in my book.

And, frankly, I don't think that thousands of tests for regexes is ridiculous. It's extremely easy to get them every-so-slightly wrong. Abigail-II is right, imho, to provide that kind of coverage. Those regexes are going to be depended on for business decisions. It might be a small part, but it's nice to know that "It Will Just Work".

And, the test suite on installation isn't always just for installation. I've played with making changes to many modules. I use their test suite for making sure my changes are backwards-compatible.

Now, maybe it might be useful for "make test" to have a "make test TEST_TYPE=dev" vs. "make test TEST_TYPE=install" (which would be the default). Then, the tester would indicate a subset of tests which are regression-only and which ones should always run.

Plus, you can install the distribution without running the test suite ...

[autarch]

Hmm, I just installed it and it only took about 60-90 seconds or so. That's not a big problem AFAIC.

[Juerd]

Now try doing it on a loaded server system. I had to upgrade R::C recently and timed the upgrade process. The exact numbers I forgot, but the tests took several minutes.