Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

RFC: Continuous Integration (CI) for your CPAN modules (for free!)

by stevieb (Canon)
on Jan 15, 2016 at 16:31 UTC ( [id://1152864]=perlmeditation: print w/replies, xml ) Need Help??

This guide will attempt to explain the basics of using a version control system (specifically GitHub), along with other free online tools to automate your builds and generate test coverage reports automatically on each update to your code.

It'll be a work in progress for some time (very high level overview at this time), so feel free to leave comments for fixes/updates etc. This doc assumes that you already have module code you've written (eg: My::Module). There's a fair amount of detail lacking because I can't remember the exact steps for everything, but over the weekend, I'll re-run the entire process and fill in the large gaps here.

First, head on over to GitHub, and sign up for a free account, and create a repository (eg: my-module).

Now, install git, and clone your newly created (empty) repo to the local machine:

git clone https://github.com/username/my-module

From the directory above your actual module code, copy the contents of the module's top level dir into the new empty repo dir:

cp -R My-Module/* my-module

Add these files to git, and perform your first check-in:

cd my-module git add . git commit -am "initial import" git push

You now have revision control for every change you make going forward to your module.

Next, head over to Travis-CI, and sign up for an account with your GitHub username. After signed in, 'sync' your account to github, and in your repositories screen, you'll see an on/off toggle for your new repository. Enable it.

Now go to Coveralls and perform the same task you did for Travis.

At this point, everything should be enabled and ready for use. To proceed, you'll need to set up a .travis.yml file in your distribution's root directory. Here's one that I use in all of my modules. It's configured to run the build on perl versions 5.8 through 5.22, and it'll also perform test coverage in prep for sending to Coveralls.

language: perl perl: - "5.22" - "5.20" - "5.18" - "5.16" - "5.14" - "5.12" - "5.10" - "5.8" os: - linux before_install: - git clone git://github.com/travis-perl/helpers ~/travis-perl-helpe +rs - source ~/travis-perl-helpers/init - build-perl - perl -V - build-dist - cd $BUILD_DIR install: - cpan-install Devel::Cover - cpan-install --deps - cpan-install --coverage before_script: - coverage-setup script: - PERL5OPT=-MDevel::Cover=-coverage,statement,branch,condition,path, +subroutine prove -lrv t - cover after_success: - cover -report coveralls - coverage-report matrix: include: - perl: 5.20 env: COVERAGE=1

Note that Travis doesn't do testing on Windows, so I always advise having a Windows machine/VM handy, and run your test suite on there to ensure it runs cross-platform (if necessary) before upload to CPAN.

Perform a change in your code, then commit and push it to GitHub. Now go to your Travis page (mine's https://travis-ci.org/stevieb9/ for example), and your build should be running or just about to start.

If all builds succeed, head on over to your Coveralls page (eg: https://coveralls.io/github/stevieb9/), and you should see a page with your repos listed, with the coverage details of your unit tests.

Click on the repository, and at the top of the page, click "Badge URLs". Copy the entire text that contains the HTML link. You'll need this, along with a Travis link for your repo to add to your POD documentation.

*Note*: I can't remember where I got my own Travis link from, but yours will be the same as mine, simply replace the name of my repo with yours. See the example below...

Now, in your POD, add a tiny bit of markup and paste in your new links. I put mine in the NAME section so it's visible immediately to visitors to my modules on CPAN:

=head1 NAME Devel::Examine::Subs - Get info about, search/replace and inject code +into Perl files and subs. =for html <a href="http://travis-ci.org/stevieb9/devel-examine-subs"><img src="h +ttps://secure.travis-ci.org/stevieb9/devel-examine-subs.png"/> <a href='https://coveralls.io/github/stevieb9/devel-examine-subs?branc +h=master'><img src='https://coveralls.io/repos/stevieb9/devel-examine +-subs/badge.svg?branch=master&service=github' alt='Coverage Status' / +></a>

This will result in two images in the NAME section of your POD when users view your module on CPAN. One will show the build success or fail, the other the percentage of test coverage you have.

Here's an example of what this looks like. You can click through both images and get to the full details of both the build process, and the coverage results.

NOTES:

  • - coveralls.io only does line testing, so I always use Devel::Cover in conjunction to get even more detailed coverage

Replies are listed 'Best First'.
Re: RFC: Continuous Integration (CI) for your CPAN modules (for free!)
by hippo (Bishop) on Jan 15, 2016 at 17:11 UTC

    Nice write-up, stevieb, thank you.

    I've seen a number of projects using coveralls.io but wasn't really sure what it gives us that Devel::Cover and cpancover don't already. Clearly cpancover only runs on dists published to CPAN, so coveralls.io gives data before publishing which might be useful, I suppose. But the keen author would already use Devel::Cover locally so I'm not really sure of the benefit of using coveralls.io for Perl projects. Could you explain a little about what the advantages are?

      Technically, there isn't a whole lot of advantage, except its automatic on each push and easy to view at a glance, and like you said, it's nice for non-CPAN modules (or releases you haven't put up yet). I could see another benefit if there's a small team working on a project, instead of everyone having to use Devel::Cover on each commit/push, they can simply go to a single website to monitor testing progress.

      I like to browse MetaCPAN every few days and peek into modules that I might like, or for issues in others modules I might want to take a crack at fixing, and I had just found it handy to have a basic coverage percent icon right up front (it shows a (very basic) indicator of effort on testing quality), so I started doing it for my modules.

      It wouldn't be worth it if it was difficult or time consuming to configure, but it's brain dead simple :)

Re: RFC: Continuous Integration (CI) for your CPAN modules (for free!)
by Mr. Muskrat (Canon) on Jan 15, 2016 at 17:27 UTC
Re: RFC: Continuous Integration (CI) for your CPAN modules (for free!)
by jeffa (Bishop) on Jan 15, 2016 at 18:57 UTC

    Very nice write up. I hope others adopt this style as well. I do have a couple of comments however.

    The first is that your travis.yml is very involved and perhaps a simpler example is less daunting up front. For example, this is what i tend to use

    language: perl before_install: - cpanm --notest Test::Pod Test::Pod::Coverage #other dependencie +s you have perl: - "5.22" - "5.18" - "5.14" - "5.8"
    Travi-CI tends to allocate 4 workers at a time, so specifying more than 4 Perls will significantly increase the time it takes to complete the entire job. Just something to consider. More here: https://docs.travis-ci.com/user/speeding-up-the-build/, but i have tried none of those examples. I am quite happy with the amount of time it takes, this is a free service after all. :)

    The second comment i have regards the adding of badges to POD. I do not recommend this. Instead I recommend adding any and all badges to your Github readme file and specify the Github repository in your distribution's Makefile:

    META_ADD => { resources => { homepage => 'https://github.com/user/Foo-Bar, repository => 'https://github.com/user/Foo-Bar.git', }, }
    ... or your Build file. Consult the docs for more. The reason i say not to put build badges inside your CPAN distros is because they reflect the state of the Github repo, not the distro on CPAN. You can also add a badge to your Github readme file that points to your (already registered) CPAN module. :)
    [![CPAN Version](https://badge.fury.io/pl/Foo-Bar.svg)](https://metacp +an.org/pod/Foo::Bar)

    jeffa

    L-LL-L--L-LL-L--L-LL-L--
    -R--R-RR-R--R-RR-R--R-RR
    B--B--B--B--B--B--B--B--
    H---H---H---H---H---H---
    (the triplet paradiddle with high-hat)
    

      Thanks for the feedback jeffa.

      I'll play around with your suggestions over the weekend and update the post if necessary.

Re: RFC: Continuous Integration (CI) for your CPAN modules (for free!)
by pryrt (Abbot) on Jun 28, 2018 at 21:51 UTC

    This meditation was my first introduction to CI, and I have have been using .travis.yml's based on this writeup for over a year, ever since I followed a link to this meditation. Recently, I found out about AppVeyor which provides free CI on Windows. I thus stole stevieb's test-brewbuild/appveyor.yml as a start... but I liked the multi-perl nature of his Travis-CI config, so I wanted to figure out how to get AppVeyor to do multi-perl as well.

    Here is what I have so far, with a description of some of my "features" or "tricks". (You can see my experimental history in https://github.com/pryrt/avtest/commits/master/appveyor.yml, or you can skip to my conclusion in the "live" example at Math-PRBS/appveyor.yml.)

    AppVeyor uses chocolatey for external package management, and that's how you get strawberry perl onto the VM. stevieb's starting config just downloaded the default strawberryperl package, but the --version option to choco / cinst allows picking from the Version History as well. I eventually came to the point where I make a matrix: of environment variables, setting the %perl% environment variable using perl: default or perl: 5.14.4.1 or similar (so that it matches the full version that chocolatey uses). For the 'default', I don't pass the --version %perl% argument, so chocolatey just installs the default 'strawberryperl' package. I kept all the strawberry versions I've tried as comments in the matrix, with notes of which ##failed## when Appveyor's chocolatey tried to install them; I haven't debugged what's wrong with those packages.

    I use a trick in the build and test sections, to run the correct make: Strawberry Perl used to use dmake, but recent versions have switched to gmake, and there was even at least one intermediate strawberry version that used dmake.exe, but also provided gmake.exe, so I couldn't just test for existence of gmake.exe. That made a pure cmd.exe version of make test impractical. I ended up spawning the one-liner perl -MConfig -le "system $Config{make}, 'test'", to use whichever was the correct make for that particular strawberry (and it should continue to work, even if future strawberry versions switched to futuremake.exe).

    I also added cover to my AppVeyor config (well, added my named-target testcover), to verify my coverage on Windows. (Originally, because coveralls only integrates with travis-ci, I didn't bother also doing coverage in AppVeyor. While writing this up, I realized having external confirmation of coverage on Windows seems nice, and provides public confirmation even before CPAN Testers Matrix is reporting for the Windows platform: I've seen it take a few days before any Windows results show up.)

    In appveyor.yml, you have to install Devel::Cover if you want to run cover (just like in .travis.yml, where you have to use cpan-install --coverage): since it's not likely in your distro's prereqs, --installdeps doesn't grab it. Just using cpanm Devel::Cover worked (usually1), but it took a long time, so I switched to cpanm --notest Devel::Cover. (Devel::Cover is a well-established module, and I am going to assume that it shouldn't fail tests on install...)

    I had experimented with the cache of the C:\strawberry directory, to save time during builds. Unfortunately, if I've read the documents correctly, AppVeyor only gives 1Gb of cache with its free accounts, and each job of each build uses up a separate portion of that cache. Since each strawberry installation is roughly 0.3Gb, that's only about 3 strawberry installations across all my projects with all their jobs before it would be full. And it sounded like it would cause builds to fail if the cache was full. Maybe my interpretation is wrong, but I can handle an extra 1-2min per job before seeing the status to ensure that it won't fail the build just because the cache was too full.

    Anyway, I hope this might be useful to somebody out there.

    -- pryrt

    PS: This post has so much chocolate 🍫 and strawberries 🍓: now I want dessert ☺.

    1: For some reason, with perl v5.18.4.1, Devel::Cover had test fails when I was first trying it with Math-PRBS, but that failure appears to have been a fluke, because it works now.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlmeditation [id://1152864]
Approved by toolic
Front-paged by kcott
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others surveying the Monastery: (3)
As of 2024-03-19 07:01 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found