Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery

Interactivity in tests - done correctly?

by tirwhan (Abbot)
on Jan 22, 2009 at 13:03 UTC ( #738141=perlquestion: print w/replies, xml ) Need Help??
tirwhan has asked for the wisdom of the Perl Monks concerning the following question:

Hello monkses,

I'm currently finishing off a module I plan to release on CPAN. It would be beneficial if the test suite included tests against a live server (simulated tests are also included, but not as good as the real thing). For this I need to prompt the user for some information, to see whether he has such a server available and what it's address etc. are.

I gather that putting interactivity into the general test suite is a big no-no (it breaks the automated smoke tests for one), so I've been thinking about how I could do this and come up with the following:

use Test::More qw(no_plan); use strict; use warnings; SKIP: { skip qq(Won't run automatically, execute "export MyModOPTRUN=yes" +to enable interactive test) unless ($ENV{MyModOPTRUN} && $ENV{MyModOPTRUN} eq "yes"); warn "\nEnter your innermost secret:\n"; my $secret = (<STDIN>); ok ($secret,"Yep, that's it"); }

Obviously replacing "MyMod" with the actual name of my module. This will produce

t/02.optional....ok 1/1 skipped: Not running automatically, execute "export MyModO +PTRUN=yes" to enable interactive test All tests successful, 1 subtest skipped. Files=1, Tests=1, 0 wallclock secs ( 0.00 cusr + 0.01 csys = 0.01 C +PU)

When run without the environment variable set, which should be all right I think.

Are there any problems with this approach? Does the "export" syntax even work on non-*NIX-shells? If not, is there an equivalent (I can test for $^O and adjust the skip message accordingly)? Is there a better way to do this?

Thanks in advance

All dogma is stupid.

Replies are listed 'Best First'.
Re: Interactivity in tests - done correctly?
by moritz (Cardinal) on Jan 22, 2009 at 13:11 UTC
    First of all I think that environment variables are the right tool for the job, although I'd call them TEST_INTERACTIVE or something like that. I'd also provide the ability to provide all information via a config file or environment variables, because the value of a test suite is that you can run it with very little effort - which gets lost if you have to enter the same things all over again.
    Does the "export" syntax even work on non-*NIX-shells?

    It doesn't even work on some Unix shells, for example on csh.

    You could just say "set the environment MyModOPTRUN variable to yes", and hope that the user know how to do it.

Re: Interactivity in tests - done correctly?
by Corion (Pope) on Jan 22, 2009 at 13:07 UTC

    Why ask the user for the login/password, when the user could set $ENV{MyModOPTRUN} to it? I would output '$ENV{MyModOPTRUN} not set, not running interactive tests' as the status message and have the instructions how to enable it in README or INSTALL.

    The "export" syntax only works in unixish shells, but somebody who wants to run the interactive tests should know how to do it for the respective OS ;)

      There is a total of three strings I need from the user, I think this is easier done interactively than having them set three different environments variables. Also, I'd like to give them some information about the nature of the test they will be running. You're right, this can be put into the README or INSTALL files, but nobody reads the damn things ;-).

      somebody who wants to run the interactive tests should know how to do it for the respective OS ;)

      True, and moritz makes an excellent point about it not even being the same on csh. I'll probably just tell them to set the environment variable and provide further assistance in the README file which they can read if they need it. Thanks!

      All dogma is stupid.

        I read the README and INSTALL files :-)

        One of the things that can piss me off to no end is watching cli cpan hang because a) the config on that machine is missing a timeout b) you didn't code one

        I would say code it, but provide a timeout, and let the thing install even if it's missing something.

        Maybe at compile time it could break and tell why.
        If your module becomes a dep for something else, maybe whomever is installing it will already be aware of what's up.

Re: Interactivity in tests - done correctly?
by mje (Curate) on Jan 22, 2009 at 13:39 UTC
      Even if you don't use ExtUtils::MakeMaker, you should use ExtUtils::MakeMaker::prompt, since every perl comes with ExtUtils::MakeMaker.

        Agreed. I slightly misread the post at first or I would have made that response use prompt.

      I am a little uncertain as what you mean.

      If I use 'prompt' in the Makefile.PL, how can the test scripts get this information.

      I am having a similar requirement as the OP, in that I need username and password to test. I do not want to use an environment variable for the password (although it is a development database, the DBA is very strict about keeping even those passwords very secure).

        Yes, as I said I initially slightly misread the question and hence the reference to Makefile.PL. You can use ExtUtils::MakeMaker->prompt in your test scripts too.

        $ perl -MExtUtils::MakeMaker -le 'print ExtUtils::MakeMaker::prompt("f +red", "default");' fred [default] # I just hit CR here default

      Thanks! I came across ExtUtils::MakeMaker::prompt when looking for answers to this question prior to posting (should have mentioned that). Correct me if I'm wrong, but I was under the impression that using E::MM::prompt means that

      a.) Anyone installing the module via Module::Build would not be prompted?

      b.) Everyone installing the module interactively (via Makefile.PL) would be prompted?

      I don't want either of these, I like to provide both a Build.PL and Makefile.PL so people have the choice of install system to use. And I also don't want to prompt every user for this information, I just want to tell the user "you can test this against against a live service if you want, and if you tell me to do so I'll prompt you for the information that's required".

      Not saying your suggestion is a bad one, but if both of these assumptions are true it's just not the right fit for me.

      All dogma is stupid.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://738141]
Approved by Corion
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (3)
As of 2018-04-21 18:01 GMT
Find Nodes?
    Voting Booth?