Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Comment on

( #3333=superdoc: print w/ replies, xml ) Need Help??

I think that increasingly we're seeing people running distribution test suites in parallel by setting the environment variable HARNESS_OPTIONS=j<n> or by executing prove -j9. chromatic discusses it in his blog here: Parallelism in Test Suites. In that post, and in the discussion that follows, module authors are encouraged to be mindful of parallelism in testing so that in the vast majority of the cases where parallelism is possible module tests are designed to facilitate that capability. An example is given where two test scripts depend on the same clean slate, and where running in parallel they stomp on each other's sandbox. In most such cases, fixing the assumptions or creating independent resources for each test script is a reasonable fix.

In a followup to the same post, aristotle makes the following statement:

Making a test suite not break under parallelism doesn’t necessitate making it run in parallel. However I believe something like say 95% of tests on CPAN will already run fine in parallel with no further ado, and of the rest, easily the majority will be very simple to fix.

In the quasi-infinitesimal remainder of cases, sure, if the effort is not worth it, just forcibly serialise the tests and move on.

I expect a push to test parallelism to require little housekeeping effort all told. There just needs to be a reliable pressure that steers the CPAN towards it.

This node is about the quasi-infinitesimal remainder of cases. syphilis maintains Inline::C, and I maintain Inline::CPP. Those modules both depend on a C/C++ compiler to do much of the heavy lifting. I don't know about all popular C/C++ compilers, but I have found that gcc doesn't seem to support parallel compiling. If two test scripts cause the C/C++ compiler to be invoked at the same time, we get a test failure. I don't think our chances are strong for getting that fixed. So the question arises, how do we, as aristotle suggests, "just forcibly serialize the tests and move on"? syphilis and I have been discussing this issue between us, and could use some enlightenment.

One thought is for each test script to set $ENV{HARNESS_OPTIONS} by stripping out the j<n> flag, and then cleaning up any ':' mess in the case of multiple flags having been set. But that won't work because by the time a test script is executed it will already be running in parallel with others; it's too late to affect the harness.

My original thought was there might be a way for Makefile.PL to cause make test to override any HARNESS_OPTIONS setting, but I'm at a loss as to how to accomplish this.

Another approach might be for Makefile.PL to detect the HARNESS_OPTIONS flags, and complain loudly before dieing. At least then someone installing a module will know why he's getting a failure (otherwise, the failures can be pretty opaque).

Does anyone know how best to deal with this sort of situation?


Other resources: Test::Harness, Controlling Test Parallelism with Prove.


Dave


In reply to Where should (or could) a distribution override HARNESS_OPTIONS? by davido

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • Outside of code tags, you may need to use entities for some characters:
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others meditating upon the Monastery: (5)
    As of 2014-12-20 13:07 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?

      Is guessing a good strategy for surviving in the IT business?





      Results (95 votes), past polls