What are all the possible ways to deploy Perl code on multiple servers?

jozef has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: What are all the possible ways to deploy Perl code on multiple servers? by saberworks (Curate) on Jul 19, 2011 at 22:05 UTC
I've done it the following two ways at different jobs. 1. We packaged perl and all deps, plus all cpan modules (and their deps) from source as RPM files that would be built on 3 different operating systems (rh enterprise linux, solaris, and osx). We shipped the RPMs to our customers when a release was cut (either on CD or electronically). It was built using make so if you changed/updated any of the package sources it would recompile it plus everything that depended on it. It was a hassle to maintain at first (I inherited it from a previous employee) but once I worked out the kinks it was very reliable and our customers liked it. It made upgrades pain-free (from their perspective). 2. At my current job we decided to use the "standard perl" on ubuntu 10.04 LTS. I mirrored the ubuntu repo and we use apt to manage all the packages across our servers. Basically we programmed against what's in the official repo and we only update our mirror after we test any package changes. The drawback here is that there are some modules that aren't in the ubuntu repo or that are too out of date to be useful. In these cases, I build custom .deb packages and put them on a separate repo (which is also included in each server's sources.list). Then there's a "my_dependencies.deb" package which gets installed that lists all our custom packages plus all the ones in the ubuntu mirror that we need. It's been working very well so far and it's much less work than maintaining all the packages ourselves. Granted, "standard perl" on Ubuntu 10.04 LTS is out of date but we don't really miss any of the new features.	[reply]
Re^2: What are all the possible ways to deploy Perl code on multiple servers? by chrestomanci (Priest) on Jul 20, 2011 at 08:41 UTC
I would be interested to know how you built the costom deb for the modules that are not in the Ubuntu repo? Did you package them nicely in a debian approved way, or was it more of an ugly package of everything, similar to how I described above? If you did make your deb packages nicely can you share some more details or scripts on how you did it?	[reply]
Re^3: What are all the possible ways to deploy Perl code on multiple servers? by i5513 (Pilgrim) on Jul 20, 2011 at 11:56 UTC
Maybe using dh-make-perl debhelper ?	[reply]
Re^3: What are all the possible ways to deploy Perl code on multiple servers? by saberworks (Curate) on Jul 20, 2011 at 14:59 UTC
I create .deb packages of each module and they're hosted on a web server with a Packages.gz file (which is generated using some command). I did it by hand (found a tutorial online) until I found dh-make-perl as i5513 mentioned. It works pretty well. For the "dependencies" deb file (the one that lists all the packages required for our servers, both from the official repo mirror and our package list) I created it by hand. Essentially you just have to create a debian CONTROL file and then run some dpkg commands on it. I can post more details when I get to work and take a look. It's one of those things that works and we only touch when we need a new module. Update: Okay here are some instructions... `$ export DEBFULLNAME="<your_name>" $ export DEBEMAIL="<your_email>"` [download] Go to search.cpan.org, search for the package, click the one you want, and right-click the Download link and click Copy Link Location. For this example I'm going to use http://search.cpan.org/CPAN/authors/id/K/KI/KILINRAX/HTML-Strip-1.06.tar.gz Make a directory for your package. `$ mkdir HTML-Strip` [download] Change to that directory `$ cd HTML-Strip` [download] Download the package `$ wget http://search.cpan.org/CPAN/authors/id/K/KI/KILINRAX/HTML-Str +ip-1.06.tar.gz` [download] Unzip the package `$ tar xvfz HTML-Strip-1.06.tar.gz` [download] Configure/build the package `$ dh-make-perl -p my-libhtml-strip-perl --build HTML-Strip-1.06` [download] Now your project directory should have the following: `drwxr-xr-x 4 4096 2010-04-13 20:34 HTML-Strip-1.06 -rw-r--r-- 1 7764 2006-02-10 11:27 HTML-Strip-1.06.tar.gz -rw-r--r-- 1 17912 2010-04-13 20:35 my-libhtml-strip-perl_1.06-1_i386. +deb` [download] Check to make sure you can install the package `$ sudo dpkg -i my-libhtml-strip-perl_1.06-1_i386.deb` [download] Make sure you can "use" the module `$ perl -mHTML::Strip -e 1` [download] Once you have the .deb package you need to host it on a web server somewhere and you need to generate a Packages.gz so apt/etc. knows about each package. Assuming you put all your .deb files in a folder called binary/ you run: `dpkg-scanpackages binary /dev/null \| gzip -9c > Packages.gz` [download] Now you have a folder called binary/ with a bunch of .deb files in and a Packages.gz with a list of those files. You can edit our /etc/apt/sources.list to add your repo like this: `deb http://<your_url>/ /` [download]	[reply] [d/l] [select]
Re: What are all the possible ways to deploy Perl code on multiple servers? by chrestomanci (Priest) on Jul 19, 2011 at 20:26 UTC
You will find that there are a lot of different solutions out there, because the problem space can be sub divided a lot of ways. For starters, how many servers do you have and how homogenous (or otherwise) are they? At one place I worked, all the servers ran the same version of Ubuntu, so it was fairly simple to use CPAN to install all the required packages on one development server, and then create a huge debian package of everying to install on the others (Using the .packlist files that CPAN helpfully writes when it installs anything). When ever we did a release of the main perl program that ran on the servers, we would install any new dependencies on the test server, and re-create the debian package. If your servers are not homogenous, could they still run the same perl code if they each had access via a shared folder or a checkout from the version control system? This should be possible if they are all running the same major version number of perl, and if any of the libaries contain XS compiled code then all the servers have the same processor archetecture and OS family.	[reply]
Re^2: What are all the possible ways to deploy Perl code on multiple servers? by jozef (Sexton) on Jul 19, 2011 at 20:51 UTC
We have all kinds of distributions in the company, but the new installations should be homogeneous - Ubuntu. It's an interesting solution converting the ".packlist" to deb. Do you know of any open source code to do this task?	[reply]
Re^3: What are all the possible ways to deploy Perl code on multiple servers? by chrestomanci (Priest) on Jul 19, 2011 at 21:22 UTC
I don't know of any open source products to automate the conversion, though there may be somthing on CPAN. As I recall I had a shell script of about 30 lines long that ran a find to get all the .packlist files, then used xargs etc to feed the list of filenames into tar in order to create a big tarball of all the perl files needed. The tarball was then unpacked into skeleton debian package directory (With a pre-prepared DEBIAN/control file). and the necessary debian specific command was run to turn the whole thing into a debian package that would unpack everything into the same locations on another server. I have probably foggotten a few details, and the process probably breaks the debian packaging policy in lots of ways, but it was a quick and dirty solution so who cares.	[reply]
Re^3: What are all the possible ways to deploy Perl code on multiple servers? by Anonymous Monk on Jul 19, 2011 at 22:06 UTC
CPANPLUS::Dist::Deb, Dist::Zilla::Deb	[reply]
Re: What are all the possible ways to deploy Perl code on multiple servers? by onelesd (Pilgrim) on Jul 19, 2011 at 17:13 UTC
If you have a small codebase or a single script, I find this to be easiest. I realize it doesn't fit all situations, but if you are the one running the scripts or you are able to use cron, this works a treat. `scp yourscript.pl user@server:/tmp && \ ssh -t user@server \ 'perl /tmp/yourscript.pl && rm /tmp/yourscript.pl'` [download]	[reply] [d/l]
Re^2: What are all the possible ways to deploy Perl code on multiple servers? by thirdm (Sexton) on Jul 19, 2011 at 21:14 UTC
ssh (or at least openssh) lets you redirect stdin over the network. So you could skip the rm here, e.g... `ssh -t user@server perl < yourscript.pl` [download]	[reply] [d/l]
Re: What are all the possible ways to deploy Perl code on multiple servers? by rurban (Scribe) on Jul 20, 2011 at 17:02 UTC
csshx - "csshX is a tool to allow simultaneous control of multiple SSH sessions" You have immediate feedback if one server behaves badly. And it's in Perl:)	[reply]
Re^2: What are all the possible ways to deploy Perl code on multiple servers? by jozef (Sexton) on Jul 21, 2011 at 06:37 UTC
Looks nice but: This software currently works on Intel/PPC OS-X 10.5 and 10.6 (32 and 64 bit). An older version is available for OS-X 10.4. I'm using "cssh" on Linux, but only now I've realized it's also written in Perl => http://search.cpan.org/perldoc?cssh. Thanks for a tip!	[reply]


Your skill will accomplish what the force of many cannot
	PerlMonks