Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

How do manage your Perl application development, build, and deployment?

by TGI (Vicar)
on Mar 17, 2009 at 21:01 UTC ( #751283=perlquestion: print w/ replies, xml ) Need Help??
TGI has asked for the wisdom of the Perl Monks concerning the following question:

I have yet to come up with a satisfactory way to manage the development, build, and deployment of my Perl applications. I'd like to hear how you have solved this problem and/or what you would like to have in an application build system that you don't have now.

Please describe your application type (is it a web app, does it run on a server, or do you bundle it using PAR or PerlApp so that you can run on perlless systems).

Key things a build system should provide:

  • Control of libraries.
    • It should be possible to check a library distribution out into my dev directory to use it in my build.
    • It should be easy to execute perl with an @INC value that will use the appropriate directories.
    • It should be possible to get a list of modules that are being sourced from the system perl install.
  • Makefile/Build integration
    • It should be easy to do a global test across the entire application by issuing only one make test or similar command.
  • Version control friendly
    • structure should not interfere with normal usage of CVS, SVN and other version control systems.
  • Cross platform
    • System should operate on Win32 and Unix derived systems at minimum.
    • Ideally, tools should function identically in all places where perl operates.
  • Single Perl install
    • It should not be necessary to install perl into a special directory as part of setting up the environment.
  • Easy start up
    • Starting an application should be a mostly automated process. Something along the lines of Module::Starter or h2xs should be available to layout a basic structure and create any standard files.

Cross-posted at: Stackoverflow


TGI says moo

Comment on How do manage your Perl application development, build, and deployment?
Select or Download Code
Re: How do manage your Perl application development, build, and deployment?
by perrin (Chancellor) on Mar 17, 2009 at 21:34 UTC
    If you search for "deployment", you'll find many discussions on this subject.
Re: How do manage your Perl application development, build, and deployment?
by saberworks (Curate) on Mar 17, 2009 at 21:57 UTC
    In my last position we had a big/complex biotech information management system. We basically packaged everything needed to run the app into RPM files and distributed those. We built RPMs for RedHat Linux, OSX, and Solaris. The set included everything - perl itself, db server, apache, all the various CPAN modules we needed (packaged individually), etc.

    To install it, there was a script on a network drive that you could run. It would install the RPMs (which all install to /usr/local/<our_company>). If you wanted a dev environment, you pass a flag and the install script would check out our code over top of the version of our code installed by RPM.

    It was clumsy at first but it fulfilled most (some?) of your requirements. Keeping all the packages updated was a major pain, but of course everything was in source control, properly tagged/branched/etc. (daily builds, release branches, etc.). Also, keeping a handle on the licenses of all third-party packages was a chore (for example, any GPL/LGPL software was off-limits and any commercial software needed a redistribution agreement).

    We also had another project where we basically provided our perl code and a list of required software/modules and left it up to the customers to install them. I'm guessing the customers liked the RPM method better because they could install the system with 1 or two simple commands.

      It seems like many of us have put together various 'works well enough, but not great' solutions. I know I've tried a few different things, but they all suck in different ways.

      Are our needs in this area too diverse for a good, common solution to emerge?


      TGI says moo

Re: How do manage your Perl application development, build, and deployment?
by sundialsvc4 (Monsignor) on Mar 18, 2009 at 12:17 UTC

    I'm sure that you will get many good responses on this ... and I second the motion that you should review the existing responses under other topic-names.

    One of the more easily-overlooked requirements, especially in the ubiquitous “shared web-host” environment, is deployment independence. The fact that you have deployed n applications, and have just updated one of them, should not affect the other (n-1). Directives like use lib make that much easier. And so does the nature of the language itself, being divided into packages and modules that you mix-and-match.

    As you observe, Perl is fundamentally a platform-independent language, so any methods that are used have to work on all platforms. Here, the “trick” is that all of that nifty platform-independent stuff must become platform-dependent somewhere, and that “somewhere” is in the form of so-called XS code. You need a compiler in order to do that ... easily done (taken for granted, in fact) on Linux, but not-so on Windows.

    (This, of course, being why “friends don't let friends drive Windows.”)   ;-D

    As far as the rest of it goes, many of these topics really do fall easily under the umbrella of “deployment,” because quite frankly I defy you to make any language or operating-system “easy” without a good installer ... and yet, once you do so, “the language being used” isn't such a critical factor anymore.

      I've read many nodes that describe diverse systems over the years, and again before I posted. I didn't see anything that does what I want. I saw a collection of ad-hoc systems, each different, each flawed. Each a duplication of engineering that is kept proprietary. The question why haven't we got a good, common solution for applications? We have a good common solution for modules.

      The only thing I've seen that looks like a step in this direction is Shipwright. I haven't had a chance to fully dig in to the docs and experiment with it, so I can't comment on how good it is.

      If there is a particular node that you think describes a great system, please post a link.

      Are our build systems our crown jewels? Do we keep them secret to gain competitive advantage? Or are they too wound up in internal politics and history to be cleaned up enough that we wouldn't be too embarrassed to share?


      TGI says moo

Re: How do manage your Perl application development, build, and deployment?
by pileofrogs (Priest) on Mar 18, 2009 at 16:46 UTC

    I've found the old unix maxim of small tools that do one job and do it well is a good one to keep in mind. A large "solution" isn't going to do exactly what you want. Instead a collection of solutions of the sub-problems of your task is more manageable. Then you can tie them together into your overarching solution.

    In other words, a not-quite-great-but-good-enough solution may be the best there is.

    --Pileofrogs

      I too find this design philosophy beneficial.

      Which tools do you use? How do you put them together?


      TGI says moo

        I use git, Module::Build and Module::Starter. I have a home-brew system for distributing the libs to my various machines. None of these are great or optimum, but it works.

        My needs are pretty simple though. I only have one OS, I don't need multiple versions of anything, I work alone, I only work on small projects.

Re: How do manage your Perl application development, build, and deployment?
by gokuraku (Monk) on Mar 18, 2009 at 19:22 UTC

    I'm not deploying to a Production environment, but I do utilize a Perl Test Harness that is distributed across a large test lab, with multiple platforms. The way we structure that is to assure that all machines are built to general specifications, including necessary compiled libraries and modules. Within the Test Harness is a Perl library that we use to keep libraries that can be utilizied across Windows and Unix platforms without worrying about compilation issues, we also include our own modules in there. This what with general requirements and our own structure we know what will be available, and have some programming standards to assure that in the case a library is unavailable there will be some graceful handling when possible.

    When I did manage deployments I tried to handle things so all packages were able to be installed easily, had everything they needed to facilitate installs, and in certain situations we had rollback capability. You want the installs to be easy and have as minimal configuration as possible, both to add into Development, QA and Production environments but to minimize downtime in the case a deployment requires bringing down Customer facing applications. I could go into more, but its probably covered here, or in other forums, at length.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://751283]
Approved by kyle
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (10)
As of 2014-09-17 16:23 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (91 votes), past polls