perl distributive "early" testing with jenkins

[melezhik]

Often when deploying application one may face the risk of divergence between testing and production environment. Even though you have a stage servers `like production one', it'd be reasonable to check distributive in production environment before release is happened. I would call it `early` testing. Yes, of course, some subtle bugs will arise only in runtime phase, and unit test cannot cover it all, I say here about prerequisite unmet issues. In perl world unit tests and prerequisites checks are executed in standart way. One follows standard procedure, when installing things.

I put here example for Module::Build based project, but with ExtUtils::MakeMaker it's almost the same:

perl Build.PL # check dependencies and generate build file
./Build 
./Build test # run unit tests

So, why not to automate this process in continues integration approach, like for example Jenkins does?

1. Upload distributive to production server
2. Unpack it
3. Run standard perl Build.PL/Makefile.PL cycle to execute unit tests and check prerequisites
4. If anything goes wrong you know it in good time, before release is happened!

So this is what I've done in jenkins perl-smoke-test plugin. Please check it out and try to use it!

Links:

• plugin home page - https://github.com/melezhik/perl-smoke-test-plugin
• download plugin here - http://repo.jenkins-ci.org/snapshots/org/jenkins-ci/ruby-plugins/perl-smoke-test/

[MidLifeXis]

Cross posted at blogs.perl.org.

[melezhik]

yes, this is mine ((: - ( By melezhik on March 18, 2013 9:58 AM under testing ), see github link - it's also is mine.

[pemungkah]

I'd like to ask a few questions just to clarify the purpose of your plugin.

The idea is that you have a tar.gz ready somewhere that is to be installed on the production server, and that you always install it via a full build cycle. (This isn't at all a bad idea; I've worked at a place that did this for their whole stack - just clarifying).

First question: why not save the tar.gz files as Jenkins build products? This would be easier than managing them yourself.

Second question: why have this plugin do the dependency checking instead of using a Makefile (even a Makefile generated by looking at the Perl dependencies)?

Third question: what's the win over a few lines of shell script to do the same thing?

[melezhik]

Hi pemungkah! There are good questions you asked. So to answer step by step ...

The idea is that you have a tar.gz ready somewhere that is to be installed on the production server, 
and that you always install it via a full build cycle. 
(This >isn't at all a bad idea; I've worked at a place that did this for their whole stack - just clarifying).
---
It's not about installing, it's about testing, it's about running smoke tests against target server 
(often it's production one, buy actually it may be any) with given distributive.

The standard work-flow is like: 

1) do commits to source control system 
2) trigger new build with jenkins
3) deploy resulted distributive on test/stage server
4) testing, checking, bug reporting ... and go to 1) 
5) releasing on production server and ... upss, sometimes facing dependencies problems 

It's okay when you do things on non production servers, but really you don't know what type of bugs you will have 
when you deploy on production, even though you test it all on stage server - *it's still not production*. 
So the idea behind the plugin is: 

1) to minimize the risks and 
2) to deal with problems on good time

It's done by early testing. "Early" because you just run some smoke test with you distributive on production, 
without doing real deploy and before release, you just want to get the answer on following questions:

- if I miss something in my dependencies?
- if my scripts pass syntax checks and don't throw dependencies related errors?

These type of bugs are so typical, so often to occur, so simple to check to not check them on pre release phase.

So early testing work-flow is like:

1) do commits to source control system 
2) trigger build with jenkins 3
4) deploy resulted distributive on test/stage server
4) check distributive on production server - if everything is wrong - report a bug - much earlier response !!!
5) testing, checking, bug reporting ... and go to 1) 
6) releasing on production server and facing much less dependencies problems - because you are have already dealt with it on 4)

First question: why not save the tar.gz files as Jenkins build products? This would be easier than managing them yourself.
---
it's not a problem. you just give a link to you distributive, which get downloaded by curl, it may be any link, 
for example link to build stored as artefact in jenkins.

Second question: why have this plugin do the dependency checking instead of using a Makefile 
(even a Makefile generated by looking at the Perl dependencies)? 
---
Indeed pluggin does it! it checks dependencies for either Build.PL/Makefile.PL distributive, 
using Module::Build or ExtUtils::MakeMaker, this is typical perl build scheme for most any perl project/distributive. 

Third question: what's the win over a few lines of shell script to do the same thing?
---
The answer is same as for the question "why people use hi level languages not assembler or C?". 
Shell is okay for rapid, simple solutions, but it's
hard to extend and modify when things get more complicated. And also jenkins pluggins provide 
you friendly GUI interface, while with bash you are only may export envars and pass parameters in script's textarea 

PS: The only one remark on early testing work-flow. In current version of plugin, the convention is that distributive 
contains all dependencies in it, so no dependencies installing happens on target server. But in the future 
I may add dependencies installing into local::lib dir even though I think it's bad idea do compiling on production server, 
jenkins CI is right place to compile/build the things.