Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Re^2: Test Survey Results

by cog (Parson)
on Aug 04, 2008 at 15:41 UTC ( #702092=note: print w/ replies, xml ) Need Help??


in reply to Re: Test Survey Results
in thread Test Survey Results

:; time make test PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_h +arness(0, 'blib/lib', 'blib/arch')" t/*.t t/01-conversion_0_10000................ok t/02-conversion_10001_999999...........ok t/03-conversion_1000000_and_up.........ok t/04-special-cases.....................ok t/71-version_numbers_are_the_same......ok t/pod-coverage.........................ok t/pod..................................ok All tests successful. Files=7, Tests=10218, 7 wallclock secs ( 0.94 usr 0.15 sys + 6.70 c +usr 0.14 csys = 7.93 CPU) Result: PASS real 0m7.180s user 0m7.726s sys 0m0.324s

There you go, over 10k tests in just 7 seconds.

Granted, that is not a full system, just a module. However, my answers remain: my largest test suite has over 10000 tests, and it runs in just a few seconds.

I also recall creating a system with ~400 tests that would run in just no time. Then again, the most enforced requisite of that platform was that it had to process over 1000 requests per second.

And it did.


Comment on Re^2: Test Survey Results
Download Code
Re^3: Test Survey Results
by ambs (Pilgrim) on Aug 04, 2008 at 15:52 UTC
    Well...
    ambs@rachmaninoff ProjectoDicionario$ time make test
    PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0)" t/*.t
    t/empty_lines....ok                                                          
    t/last_line......ok                                                          
    t/test_terms.....ok                                                          
    All tests successful.
    Files=3, Tests=67546, 16 wallclock secs (13.43 cusr +  1.40 csys = 14.83 CPU)
    
    real	0m15.786s
    user	0m18.537s
    sys	0m1.806s
    

    This is not to show tests run quickly, as the tests are quite simple. What I would like to say is that these tests in particular are not testing code, but testing data quality (a PT dictionary formatting).

    Why is this helpful? Because when I too the time to write the tests, I found out that I had more than 6000 entries with formatting errors.

    Alberto Simões

Re^3: Test Survey Results
by Anonymous Monk on Aug 04, 2008 at 16:30 UTC
    Obviously one can have a ton of trivial tests that run in very little time. That doesn't demonstrate a lack of bias. The bias was not contemplating the possibility that an extensive test suite could take much more than 20 minutes to run. In my experience, extensive test suites on a company-wide scale usually take quite a bit more than 20 minutes to run, so I don't consider such to be an 'outlier' as characterized by Ovid to dismiss the criticism. But I'd not have set the upper limit on number of tests at 10,000 either. I'm sure we each have over 100,000 tests at my current and previous employer.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://702092]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (8)
As of 2014-08-30 08:11 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (291 votes), past polls