friedo has asked for the wisdom of the Perl Monks concerning the following question:

I have a web application with a "click here if you forgot your password" feature. When clicked, it generates a new, random password and emails it to the user. It's time to write some unit tests.

I can test that the email gets properly generated by using Test::MockObject to make a mock MIME::Lite so I don't have to really send an email. However, I can't think of a good way to test the password-generating function itself.

The random password function is quite simple. It picks a random word from a file and then sticks a random three-digit number on the end. How do I write a test that can tell the difference between a "good" random result, and something erroneous? I could regex the result and check if it looks right, but that seems like a cop-out.

Lest you think this concern is trivial, it is only a small example. I have several other non-deterministic behaviors which are more complex and do require some more rigorous testing. I wanted to start with this example so I have some ideas about how to proceed with the harder ones.

  • Comment on Unit Testing Non-Deterministic Behaviors

Replies are listed 'Best First'.
Re: Unit Testing Non-Deterministic Behaviors
by Zaxo (Archbishop) on Jun 11, 2007 at 04:08 UTC

    You can test that the generated password has the form you expect. Suppose you have read and chomped the wordlist into the keys of a hash, %words. Then the expression,

    $passwd =~ /^(\w+)\d{3}$/ and exists $words{$1}
    is true only if the password has the form you describe.

    After Compline,

Re: Unit Testing Non-Deterministic Behaviors
by tirwhan (Abbot) on Jun 11, 2007 at 08:58 UTC

    This depends on how you pick the word from the file and create the number, but if you use rand for both Test::MockRandom may be a good solution for you. Essentially, this allows you to override rand in the tested module, and make it return known results which you can test for.

    Just as a note, IMO a random word from a file with a number stuck at the end is a pretty horrible password generator and will almost certainly produce some easily guessable passwords over time. This is in addition to the oft-repeated mistake of sending a cleartext password through unencrypted mail. It would be better to generate a random string to send to the user for logging on once to enter a password of his own (advising some best practices on password generation on the same page).

    All dogma is stupid.

      Semi-OT, but PLEASE, read tirwan's second paragraph over ...and over again!!! And then, if necessary, read up on dictionary-based attacks.

      Yes, I noted your comment that this is preliminary and hope that it means you really have no intention of doing the job this way, but -- even if so -- to future casual readers, please re-read and heed tirwan's second paragraph again.

      Unless the "words" in the input file are themselves random strings, using them and appending three digits is about as safe as using the "Captain Midnight decoder ring" that I so desired, ca. 1948... back when it cost a boxtop from Chex (which I don't think were called Chex, then) and a dime (no self-addressed envelope required).

      Thanks, tirwhan, I think Test::MockRandom will be helpful in this case. And yes, the user is required to change their password to something good after logging in with the temporary password.
        ..the user is required to change their password to something good after logging in with the temporary password.

        Great, but in that case it is even less necessary to generate the new password using existing words. The only valid rationality for using passwords containing real-language words is that the user is more likely to remember them and less likely to write them down and stick them to his monitor or store them on his computer somewhere. Since your newly generated password is only used once (to allow the user to set his own pw), there is no need for him/her to remember it and it might as well be a jumble of random letters.

        All dogma is stupid.
Re: Unit Testing Non-Deterministic Behaviors
by GrandFather (Saint) on Jun 11, 2007 at 04:06 UTC

    Generate a bunch of passwords and check the frequency for each one. Since you (presumably) know how may words there are and you know the number of digits in the number, you ought be able to set a sensible theoretical threshold for the likelihood of generating duplicates of passwords (or password parts if you want to deal on that level).

    If you want to work a little harder you could look at letter frequency in the word set and letter frequency in the generated test set and check that they are similar.

    DWIM is Perl's answer to Gödel
Don't reinvent the wheel
by nferraz (Monk) on Jun 11, 2007 at 16:53 UTC

    A little off-topic, but have you looked for a password generator in CPAN? You'd find interesting stuff, like Crypt::RandPasswd, a random word generator for pronounceable passwords.