http://www.perlmonks.org?node_id=491553

tomhukins has asked for the wisdom of the Perl Monks concerning the following question:

I've written some simple tests with Test::WWW::Mechanize that check a Web site behaves as it should.

I am running these tests within cron, which sends me messages that look like:

1..8 ok 1 - The object isa Test::WWW::Mechanize ok 2 - Mech object created ok 3 - Submit bad password isa HTTP::Response ok 4 - Correct password welcomes us ok 5 - Bad password notified as a failed login ok 6 - Submit correct username and password isa HTTP::Response ok 7 - Correct password welcomes us ok 8 - Correct password not a failed login

If I alter cron to run the test script through prove I get briefer output:

test-dedicated....ok All tests successful. Files=1, Tests=8, 2 wallclock secs ( 0.16 cusr + 0.36 csys = 0.52 C +PU)

Ideally, I would like cron to only send mail if any tests fail. I could write a wrapper script to deal with this, but I wonder if you can think of a neater way?

I also wonder if I can easily report when the script didn't run at all. In this case, no tests will fail, but I still care about what happened.

Over time, I expect to add more tests in different scripts. For a collection of tests, does it make sense to write a Makefile.PL or can I get away with something like prove /path/to/*.t?

I'm looking for a clean and simple approach. Tools like Nagios look helpful, but seem like overkill for this problem to me. Any thoughts?

Replies are listed 'Best First'.
Re: Reporting test failures within cron
by davidrw (Prior) on Sep 13, 2005 at 12:23 UTC
    Assuming that whatever you're running (i think prove does) uses exit codes on failure, something simple like this would work (bash syntax):
    0 3 * * * prove foo.t > /tmp/prove$$ 2>&1 || cat /tmp/prove$$
    And it relies on cron's behavior of emailing you whatever output is generated by a job -- here we're printing the whole output iff it failed..
      This works but for security reasons it would be better to do something like: (tmp_file_name=`mktemp`; prove foo.t > "$tmp_file_name" 2>&1 || cat "$tmp_file_name")
Re: Reporting test failures within cron
by Hue-Bond (Priest) on Sep 13, 2005 at 12:30 UTC
    I am running these tests within cron
    Ideally, I would like cron to only send mail if any tests fail.

    cron only sends mail when output is generated. What you want is that your tests be silent if everything goes ok (ie: change your print statements or redirect STDOUT) and direct error messages to STDERR. That way, you can control whether you want all the output (by not redirecting STDOUT) or only errors (ideally none -> no output -> no mail from cron).

    --
    David Serrano

      That's true, but as I'm using Test::WWW::Mechanize with Test::More I don't output anything myself directly: the testing tools do this for me.
Re: Reporting test failures within cron
by chromatic (Archbishop) on Sep 13, 2005 at 18:27 UTC
Re: Reporting test failures within cron
by adrianh (Chancellor) on Sep 17, 2005 at 20:54 UTC
    Ideally, I would like cron to only send mail if any tests fail. I could write a wrapper script to deal with this, but I wonder if you can think of a neater way?

    Use a script that somebody wrote earlier :-)