in reply to Re^3: How to merge two different coverage reports remotely
in thread How to merge two different coverage reports remotely

But generating a test coverage report depends on the codebase, not the environment, unless you're using it in some way I've not encountered.

I hadn't chimed in because I've never tried merging the coverage report. But I can think of lots of situations where the environment will influence which code can be covered in a given environment, and thus will influence the test coverage report.

Here are three examples:

# the coverage report on MSWin32 will be different on Win10 from Ubunt +u Linux. # I assume the OP would like to merge the two reports to make sure tha +t all the # functions are covered at least once between the two (or more) differ +ent # environments if($^O eq 'MSWin32') { run_this_function(); } else { run_a_different_function(); } # or # the coverage report is different in the environment where # MY_ENV_VAR is set true vs where it's false or undefined, # and the OP would like to merge the two reports to make sure that all + the # functions are covered at least once between the two (or more) differ +ent # environments if( $ENV{MY_ENV_VAR} ) { run_special_case(); } # different parts of the code are covered depending on whether some ot +her # executable is found, so that it can be run if( -x '/x/y/bin/my_executable' ) { my $ret = qx(/x/y/bin/my_executable); # cannot run on system X, wher +e repo is in /a/b/repo/run.pl, and /x/y/ hierarchy might not even exi +st! ... error processing ... $some_outer_variable += $ret; }

In all of these situations, on system X the coverage will show certain functions or blocks of code run, but others not; and system Y may have a different group of functions or blocks that are run. So for example, maybe each system can run the same 80% base coverage, but system X covers 5% of code blocks that cannot run on Y and system Y has 15% of code blocks that cannot run on X; the individual reports say 85% coverage on X and 95% coverage on Y, when in reality, together they make 100% coverage, but neither report shows that. So the OP is asking how to merge those so that they can tell whether or not all lines of code are covered in at least one of the test systems.

Replies are listed 'Best First'.
Re^5: How to merge two different coverage reports remotely
by 1nickt (Abbot) on May 26, 2021 at 14:20 UTC

    Personally, I would want to mock the environment differences and know that the tests were passing and the code covered before deploying.


    The way forward always starts with a minimal test.
      mock the environment differences before deploying

      I'm always afraid that something will be missed while mocking, or I'd get something wrong. So if I can run my tests on a system that's more like what the end user will have, I like to use it. For example, GitHub Actions, Travis-CI and AppVeyor allowing for testing on different OSs or on different environments with the same OS, even before you've deployed to the end-user machines. And in situations like that, I can see why merging the coverage reports would be nice (whether for verifying actual coverage, or for verifying that the mocked environment works the same as the real environment.)

      If mocking is sufficient for you, and you don't need to merge coverage reports; great. I still think it's an idea worth pursuing for how I understand the OP's situation.