Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

Best way to build dependencies in a CI?

by Dallaylaen (Hermit)
on Apr 10, 2019 at 06:16 UTC ( #1232381=perlquestion: print w/replies, xml ) Need Help??
Dallaylaen has asked for the wisdom of the Perl Monks concerning the following question:

Hello dear esteemed monks,

We have a project with a lot of CPAN dependencies. So we need to install them, meeting the following conditions:

  • Exactly the same set of modules is installed on all of our production servers;
  • The developers are able to add dependencies at will, without ops interaction (after code review, of course);
  • The build happens at every commit and it doesn't take forever.

What we currently do is:

  • unpack dependency cache;
  • install missing dependencies with cpanm;
  • run test suite against the updated library;
  • update dependency cache;
  • pack libraries together with the product itself into a huge tarball;
  • distribute the tarball to the servers, if needed.

What I'm currently pushing for is to build the dependency cache from scratch from time to time (say weekly) and only install the missing modules (we don't add new dependencies every day after all). This would increase the resilience to changes in upstream and also allow to purge unused dependencies.

But maybe there's a better way to achieve the stated requirements?

  • Comment on Best way to build dependencies in a CI?

Replies are listed 'Best First'.
Re: Best way to build dependencies in a CI?
by afoken (Abbot) on Apr 10, 2019 at 20:38 UTC

    Maybe a local CPAN mirror could help? You could update the mirror only when you really want updates (instead of updating daily or hourly). Then, point all cpan(m) instances to the local mirror.

    Or, if all servers and dev machines use the same OS, build an OS-specific package (*.deb, *.rpm, whatever ...) with the required modules, and use the OS to install and update the packages from a local package repository.

    Alexander

    --
    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
      Well I'm thinking of both, leaning towards the second. Also thinking about trying out NIX package manager as we don't really need tight integration with the underlying system.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1232381]
Approved by Athanasius
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (4)
As of 2019-04-20 10:40 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    I am most likely to install a new module from CPAN if:
















    Results (108 votes). Check out past polls.

    Notices?
    • (Sep 10, 2018 at 22:53 UTC) Welcome new users!