Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Fast coverage module or hack

by vsespb (Chaplain)
on Dec 02, 2019 at 06:42 UTC ( #11109528=perlquestion: print w/replies, xml ) Need Help??

vsespb has asked for the wisdom of the Perl Monks concerning the following question:

Hi.

I have the following system running: It executes tests for large project on CI server with code coverage (with Devel::Cover ) turned on, after that it creates map telling which test executes which module. I.e. if, when running test1.t and modules Mod1.pm and Mod2.pm have non-zero statement coverage, it add record to map: test1.t(Mod1.pm, Mod2.pm).

Next, that map is used to determine which tests to be ran in next CI run. I.e. if I have record test1.t(Mod1.pm, Mod2.pm), that means when Mod1.pm modified, then test1.t should be ran, and if Mod3.pm modified test1.t should NOT be run. This way I run fewer tests each time and save lots of time running tests.

Question is: Devel::Cover is too slow, I need to find thing which is faster than Devel::Cover, but still fit my need. It can be as simple as:

  • Devel::Cover hack which stop collecting stats (and start work faster) for the file after determining that the file has non-zero coverage.
  • Some other module to get info that "some" code is executed from module X during run (except code in "BEGIN" blocks).

Any ideas?

Replies are listed 'Best First'.
Re: Fast coverage module or hack
by tobyink (Canon) on Dec 02, 2019 at 07:50 UTC

    First idea: if you load modules on demand — require at run-time instead of use at compile-time — then %INC will tell you what modules each test script has tested.

    Second idea: the map probably doesn't change that often, so you don't need to update it every check-in. Check when it was last updated, and only run your test suite with Devel::Cover on to update the map if the map is stale by more than X hours. (X might be 6?) After a major refactoring, you might want to manually re-trigger the map to be regenerated.

      First idea - no. we use "use" and compile time to load all stuff before "fork" webserver and to save memory with COW, so every module "use" maybe 80% of other modules.

      Second idea - yes we already do this. We run map collection once per day (before Devel::Cover is so slow for us). But when map changes, it actually can break things for rest of day. So we run map collection once per day, but also do git-diff to the commit where map collection was made, then there is no possibility of use of wrong map then, but diff gets bigger during the day. So we're looking how to speed up coverage module.

        How slow are we talking about? 10 minutes? An hour? 6 hours?

        Would doing the Devel::Cover run overnight instead of the day help?

        I think I have an idea for a solution but it would require a one-line change (a use statement added) to each module. Does that sound acceptable?

        My solution will need me to do a little work to figure out the details, but it seems like an interesting task to work on, and it might help me with one or two of my own projects.

        Edit: actually probably don't need a change to each module.

        Edit again: actually do.

Re: Fast coverage module or hack
by Anonymous Monk on Dec 02, 2019 at 07:26 UTC

    Any ideas?

    Stop testing ? Yup

    If Devel::Cover is too slow, you're using it way too much

    Just how fast can you write code anyway that Devel::Cover is too slow?

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://11109528]
Approved by marto
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (6)
As of 2019-12-05 22:32 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Strict and warnings: which comes first?



    Results (153 votes). Check out past polls.

    Notices?