Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid

Comment on

( #3333=superdoc: print w/replies, xml ) Need Help??

I think that increasingly we're seeing people running distribution test suites in parallel by setting the environment variable HARNESS_OPTIONS=j<n> or by executing prove -j9. chromatic discusses it in his blog here: Parallelism in Test Suites. In that post, and in the discussion that follows, module authors are encouraged to be mindful of parallelism in testing so that in the vast majority of the cases where parallelism is possible module tests are designed to facilitate that capability. An example is given where two test scripts depend on the same clean slate, and where running in parallel they stomp on each other's sandbox. In most such cases, fixing the assumptions or creating independent resources for each test script is a reasonable fix.

In a followup to the same post, aristotle makes the following statement:

Making a test suite not break under parallelism doesn’t necessitate making it run in parallel. However I believe something like say 95% of tests on CPAN will already run fine in parallel with no further ado, and of the rest, easily the majority will be very simple to fix.

In the quasi-infinitesimal remainder of cases, sure, if the effort is not worth it, just forcibly serialise the tests and move on.

I expect a push to test parallelism to require little housekeeping effort all told. There just needs to be a reliable pressure that steers the CPAN towards it.

This node is about the quasi-infinitesimal remainder of cases. syphilis maintains Inline::C, and I maintain Inline::CPP. Those modules both depend on a C/C++ compiler to do much of the heavy lifting. I don't know about all popular C/C++ compilers, but I have found that gcc doesn't seem to support parallel compiling. If two test scripts cause the C/C++ compiler to be invoked at the same time, we get a test failure. I don't think our chances are strong for getting that fixed. So the question arises, how do we, as aristotle suggests, "just forcibly serialize the tests and move on"? syphilis and I have been discussing this issue between us, and could use some enlightenment.

One thought is for each test script to set $ENV{HARNESS_OPTIONS} by stripping out the j<n> flag, and then cleaning up any ':' mess in the case of multiple flags having been set. But that won't work because by the time a test script is executed it will already be running in parallel with others; it's too late to affect the harness.

My original thought was there might be a way for Makefile.PL to cause make test to override any HARNESS_OPTIONS setting, but I'm at a loss as to how to accomplish this.

Another approach might be for Makefile.PL to detect the HARNESS_OPTIONS flags, and complain loudly before dieing. At least then someone installing a module will know why he's getting a failure (otherwise, the failures can be pretty opaque).

Does anyone know how best to deal with this sort of situation?

Other resources: Test::Harness, Controlling Test Parallelism with Prove.


In reply to Where should (or could) a distribution override HARNESS_OPTIONS? by davido

Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":

  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?

    What's my password?
    Create A New User
    and all is quiet...

    How do I use this? | Other CB clients
    Other Users?
    Others browsing the Monastery: (6)
    As of 2017-11-20 01:09 GMT
    Find Nodes?
      Voting Booth?
      In order to be able to say "I know Perl", you must have:

      Results (282 votes). Check out past polls.