Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine

Re^2: Test Survey Results

by cog (Parson)
on Aug 04, 2008 at 15:41 UTC ( #702092=note: print w/replies, xml ) Need Help??

in reply to Re: Test Survey Results
in thread Test Survey Results

:; time make test PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_h +arness(0, 'blib/lib', 'blib/arch')" t/*.t t/01-conversion_0_10000................ok t/02-conversion_10001_999999...........ok t/03-conversion_1000000_and_up.........ok t/04-special-cases.....................ok t/71-version_numbers_are_the_same......ok t/pod-coverage.........................ok t/pod..................................ok All tests successful. Files=7, Tests=10218, 7 wallclock secs ( 0.94 usr 0.15 sys + 6.70 c +usr 0.14 csys = 7.93 CPU) Result: PASS real 0m7.180s user 0m7.726s sys 0m0.324s

There you go, over 10k tests in just 7 seconds.

Granted, that is not a full system, just a module. However, my answers remain: my largest test suite has over 10000 tests, and it runs in just a few seconds.

I also recall creating a system with ~400 tests that would run in just no time. Then again, the most enforced requisite of that platform was that it had to process over 1000 requests per second.

And it did.

Replies are listed 'Best First'.
Re^3: Test Survey Results
by ambs (Pilgrim) on Aug 04, 2008 at 15:52 UTC
    ambs@rachmaninoff ProjectoDicionario$ time make test
    PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0)" t/*.t
    All tests successful.
    Files=3, Tests=67546, 16 wallclock secs (13.43 cusr +  1.40 csys = 14.83 CPU)
    real	0m15.786s
    user	0m18.537s
    sys	0m1.806s

    This is not to show tests run quickly, as the tests are quite simple. What I would like to say is that these tests in particular are not testing code, but testing data quality (a PT dictionary formatting).

    Why is this helpful? Because when I too the time to write the tests, I found out that I had more than 6000 entries with formatting errors.

    Alberto Simões

Re^3: Test Survey Results
by Anonymous Monk on Aug 04, 2008 at 16:30 UTC
    Obviously one can have a ton of trivial tests that run in very little time. That doesn't demonstrate a lack of bias. The bias was not contemplating the possibility that an extensive test suite could take much more than 20 minutes to run. In my experience, extensive test suites on a company-wide scale usually take quite a bit more than 20 minutes to run, so I don't consider such to be an 'outlier' as characterized by Ovid to dismiss the criticism. But I'd not have set the upper limit on number of tests at 10,000 either. I'm sure we each have over 100,000 tests at my current and previous employer.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://702092]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (4)
As of 2017-03-01 18:50 GMT
Find Nodes?
    Voting Booth?
    Before electricity was invented, what was the Electric Eel called?

    Results (428 votes). Check out past polls.