Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

comment on

( #3333=superdoc: print w/replies, xml ) Need Help??

There's a little bit of an XY Problem here. Your task is to "organize a framework for cooperation between the Quality Assurance department and the community". But your questions are about the Perl testing infrastructure, which has little to do, directly, with community involvement.

In my view, the Perl testing infrastructure is effective for several reasons:

  • Running tests was made part of the module installation cycle: make; make test; make install

  • Writing tests was made easy, through a good framework (Test::Simple, Test::More and friends) and good instructions (e.g. Test::Tutorial).

  • Lots of evangelism and infrastructure support (as noted in other responses).

However, this is all about module authors writing tests, not about community feedback into the testing process. I count myself lucky to have bug reports that include a patch -- and I almost never get a bug report that includes a test file demonstrating the bug. I wrote "The value of test-driven bug reporting" to encourage more of it.

So, back to your task -- coordinating QA and community -- I think you need to look beyond the Perl testing infrastructure. You need to look at collaborative projects -- perhaps in Perl, perhaps elsewhere -- and see what works and what doesn't.

For example, there was/is the Phalanx project. Here in NY, the local Perl group collaborated to work on two Phalanx modules, and had a devil of a time getting their work incorporated back into the modules by the author. That's led to a general disinterest in repeating that process. People want some sort of sense of feedback and accomplishment from their work.

I think a better example is Pugs -- audreyt's Perl 6 interpreter. Commit bits are handed out freely. And there's a real emphasis on automated testing with something like 18,000 tests (if I recall correctly). Look at the Smoke Reports -- in particular, drill into the details and look at some of the graphical test output. This may be a model to emulate.

For your task, I offer these suggestions:

  • Put your test suite into a repository and give out commit bits liberally. Make it easy for people to contribute tests.

  • Create automated smoke tests against the repository so people can quickly see that their tests are included and what the results are.

  • Write good, easy documentation to help people get started writing tests in your framework.

  • If using Perl, consider tools like Perl::Critic to help identify contributed tests with poor style that your QA department should clean up. Don't use it as a gateway to make contributions harder, just use it to QA your QA.

  • Track and publicize contributions: aim for reputation-based reward rather than monetary or other awards. (Look at the XP system on Perl Monks for example.)

I hope this sparks some useful thinking. Best of luck.

-xdg

Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.


In reply to Re: Creating a co-operative framework for testing by xdg
in thread Creating a co-operative framework for testing by gmax

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (2)
As of 2021-08-04 18:24 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My primary motivation for participating at PerlMonks is: (Choices in context)








    Results (42 votes). Check out past polls.

    Notices?