Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Pedantically thorough one-shot catalyst sandbox installation, soon it will automagically set up demo apps too...

by tphyahoo (Vicar)
on Dec 13, 2006 at 09:27 UTC ( #589529=perlmeditation: print w/ replies, xml ) Need Help??

I have released a catalyst one-shot installer: pimpmycat -- into Google code project hosting.

This is the way I have evolved to manage complexity in perl projects that have a lot of dependencies. For me, the Catalyst ecosystem is the quintissential example of this, but I am sure it is not the only one, and I think this approach could be used in other contexts.

I am curious what the other monks think of this, both those involved with catalyst and those not. The README documentation is pasted in below.

Pimpmycat is the easy way to install a sane catalyst development sandbox on a
possibly non-root-account virgin shell. It provides the developer, or the
curious newbie who would like to experiment, with a more or less "one click" way to
install a freshly compiled perl, an optimized build system (with an updated CPAN
and Module::Build), Catalyst, many Catalyst helper modules, and finally a number
of Catalyst tutorials and sample applications, which should already be configured
and ready to run. (This last item is still a work in progress, but that is what
I'm shooting for.)

My hope is that this will ultimately lead to a slick catalyst demo installer
that will run without any babysitting at all. Run it overnight, come back in the morning
and the final message of the installer is various urls that you can open in your web browser
to view the various sample and tutorial apps. Not quite there yet, but I I think I'm close.

Pimpmmycat is ideally suited for users who would like to try out Catalyst on a
shared web host, where there is a large amount of hard disk space but no root access.
It will also provides a useful tool to help CPAN authors ensure that their distributions
install cleanly in various scenarios. (EG, a virgin perl install; a virgin perl with an updated
CPAN, a virgin pwel plus updated CPAN and Module::Build, etc.)

Basically, this is a collection of simple shell scripts. The main
function is to install a collection of external dependencies organized
around perl under a single top level directory. Along with this, there
are user friendly backup and revert helper scripts to snapshot all
external dependencies at any particular point in time. Snapshot backups
are taken during the install at various crucial points. So, for example, you don't have
to sit through the half-hour perl compile process more than once. The next
time you want to verify if something installs cleanly against a particular
configuration, you can snapshot-revert to it in just a few seconds.

Developers are encouraged to modify the install script to fold in the particular
dependency idiosynchracies that their individual projects may have.

The motivating idea is that ideally a catalyst web project (or
actually any project) should be re-installable at will on a virgin
system without any babysitting. However, various rough edges in the
perl ecosystem make this impracticable. Compiling and installing
takes a long time, and many important CPAN modules are "evil" in the
sense that they ask for user input, even if only to cocnfirm some
default. Pimpmycat tries to minimize this evil.

Also, I believe that these constraints tempt module authors to release
modules which, for example, pass all tests only when some particular
dependency is there that they have forgotten about. By providing an easy
way for module authors to snapshot-revert all the way back to a fresh perl,
more thorough testing is encouraged.

This approach could be helpful in non-catalyst development scenarios.
I have toyed with the idea of generalizing this concept as "Pimpmyperl" or
something along those lines, but I explicitly oriented myself around catalyst
for two reasons. First, I am doing a lot of catalyst development now.
Second, the large number of CPAN dependencies required to do useful work
in the Catalyst ecosystem are its greatest strength, but in my opinion also
its achilles heel. So, this is my humble offering to strengthen the strengths
and work around the weakness. Or maybe I just liked the sound of "pimpmycat" :)

The pimpmycat installer has been tested to death on a linode user-mode
virtual linux server running ubuntu dappper drake.

To actually get started using it, do as follows.

Checkout pimpmycat from google project hosting

svn checkout http://pimpmycat.googlecode.com/svn/trunk/ pimpmycat

Configure environment

edit the

$EXTERNAL_DEPENDENCIES_TOP

variable exported by

pimpmycat/environment/environment.sh

to conform with your home directory.

This config file is sourced by both the one-shot install script,
and the run environment.

Build external dependencies

thartman@linodewhyou:~/pimpmycat/install$ ./install.sh

-- This is a very thorough install script; takes a long time (maybe hours) to complete
It installs a local perl, updates cpan, installs module build, installs catalyst,
installs catalyst helpers, installs database programs, creates sample databases,
and finally installs various catalyst demo apps which
should work out of the box. (the last three items are a work in progress)
This script requires some babysitting, but it gives many hints to make the installation as user friendly as possible
If it doesn't work the first time, there is lots of documentation inside this script
on how to troubleshoot, work with, and improve it.
$ ./install.sh

Get to know the working environment environment you just installed

-- your default perl... don't like it:

thartman@linodewhyou:~/pimpmycat/install$ which perl # who knows what has been installed here
/usr/bin/perl

-- the perl you get after giving yourself an environment,
which you have complete control over even if you are not root:

thartman@linodewhyou:~/pimpmycat/install$ . ../environment/environment.sh
thartman@linodewhyou:~/pimpmycat/install$ which perl
/home/thartman/pmc_external_dependencies/current/localperl/bin/perl

-- all external dependencies controlled by pimpmycat are installed under

~/pmc_external_dependencies/current

thartman@linodewhyou:~/nbarter/install$ ls -lth /home/thartman/pmc_external_dependencies/current/
localperl
localperl_build
instantcrud_svn_co
etc.....

-- Everything installed into this directory is backed up at various points during the install script
and can be reverted at will, and conveniently, using utilities described below


Get to know your backups

ls ~/pmc_external_dependencies/backups/

gets something like

external_dependencies_current-cpanconfigured-2006-12-08-0407.tar.gz
external_dependencies_current-perlbuilt-2006-12-07-0431.tar.gz
external_dependencies_current-modperlinstalled-2006-12-07-0445.tar.gz

..... more backups from various points in the install

Get to know your backup and revert utilities

If something went wrong with the install, or if you just feel like experminting,
you can revert your perl development sandbox to any point that has been archived here.

For instance, if you want to test that a CPAN module you have under development installs cleanly against a
freshly installed perl (where CPAN module hasn't been updated, and so is a few versions old, and you
don't have Module::Build), you could revert your current perl to the archive like so:

thartman@linodewhyou:~/pimpmycat/install$ . functions.sh # source the backup/revert functions for your environment
thartman@linodewhyou:~/pimpmycat/install$ revert_external_deps ~/whyou_external_dependencies/backups/external_dependencies_current-perlbuilt-2006-12-07-0431.tar.gz # revert it

Or, if you would like to archive your current "external dependencies" to a particular point in time
that you think you may want to later revisit, do:

thartman@linodewhyou:~/nbarter/install$ backup_external_deps hellokitty
backing up /home/thartman/whyou_external_dependencies/current dir into /home/thartman/whyou_external_dependencies/backups/external_dependencies_current-hellokitty-2006-12-12-0406.tar.gz

The backup/revert commands are just simple tar commands with a bit of tweaking to work nicely with pimpmycat.
Look inside functions.sh if you are curious how this works

Set up database ( To do: change this to work with sqlite, which is easier to install and which tutorials work with )

-- Need root

$ ./postgres-setup.sh

See if it worked

$ ./runit.sh # should start the catalyst server for the sample application included in the catalyst tutorial (haven't gotten this working yet)

$ wget http://localhost:3000 # should fetch the home page of the sample application

If there are problems with this last bit, the install.sh script didn't do what it was supposed to do.

I have done my best to package pimpmycat to work on non root systems, but I may have overlooked something.
So first off have a look at the system dependencies in

install/apt-get-installs.sh

This is the script I run against a freshly installed ubuntu system to
get things bootstrapped. I have tried to only include packages which
will be included by default on a shell account provided by a
reasonable web hoster, but maybe I overlooked something. So, verify
that everything this script would install is in fact installed for
you. If there's something missing and you don't have root access,
you could ask your sysadmin if they could just run it as root,
since nothing too controversial gets installed.

If that doesn't seem to be the problem, look at the output files
written under /pimpmycat/install and see if you can diagnose things
from there.

Bug reports, improvements, patches, and suggestions can be made to

thomashartman1@gmail.com

or the catalyst mailing list.

Known issues:

Pimpmycat been tested on a virtuozzo virtual linux at tectonic.net hosting, and found to have problems:
Cache::FastMmap won't install because of system libraries issue.
This isn't a problem with user mode virtual linus.
I would like to know if it is a problem with xen virtual linux.

Comment on Pedantically thorough one-shot catalyst sandbox installation, soon it will automagically set up demo apps too...
Re: Pedantically thorough one-shot catalyst sandbox installation, soon it will automagically set up demo apps too...
by Anonymous Monk on Jun 13, 2007 at 07:36 UTC

    First, I'd like to say thank you very much for creating this installer. I especially like the whole idea of completely sandboxing the installation and having a backup/revert function. I've finally got it to install completely, but unfortunately it took much more guiding than I would have hoped. (I spent about 4 hours fixing things.)

    For some reason installing modules from CPAN kept failing--some dependencies were forgotten, and I had to stop the process, launch CPAN manually, install the missing module, and continue the install again with "perl ./cpan-mods.pl".

    I basically went through the install.sh script manually, executing the instructions at the command line one at a time, fixing problems as they occurred. I ran into some tests failing with a message saying that the "output looks okay", which doesn't sound like an error at all, and others where the test scripts try to listen for a local network connection. For these I forced the module to install.

    I realize that there's not much that can be done to fix these things in the pimpmycat script itself, because these hundreds of dependency modules are constantly changing and breaking. It's basically dependency hell with Perl modules. Does anyone have a solution to this situation?

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://589529]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (4)
As of 2014-07-12 14:58 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    When choosing user names for websites, I prefer to use:








    Results (240 votes), past polls