Re: Truly randomized keys() in perl 5.17 - a challenge for testing?

[saintmike]

Let me clarify: The module referenced in the original posting has a function hash_serialize() that takes a reference to a hash like { a => "b", c => "d" } and turns it into something like "a=b&c=d" or "c=d&a=b", depending on what keys() returns underneath:

https://github.com/mschilli/php-httpbuildquery-perl/blob/master/HTTPBuildQuery.pm#L63

Now how am I supposed to test that the outcome is what I'd expect? With two hash elements, you could argue that I could generate all permutations of possible result strings and check if one of them matches the one I got by running the function, but with thousand entries this becomes unwieldy.

The general problem is this: I have an unpredictable function (keys()) and its result gets processed, and the processed result needs to get checked against an expected outcome.

Unless keys() can be switched to a test mode to produce predictable results (doesn't need to be sorted, just give me something predictable, like before perl 5.17), what are the options?

By the way, some people have suggested to use "sort keys" in my algorithm every time, but putting in extra complexity (sort) at runtime to make sure I can test perfectly fine code is just plain wrong.

[chromatic]

Now how am I supposed to test that the outcome is what I'd expect?

Instead of hardcoding a serialized string and testing for an exact match, deserialize the results and test with cmp_deeply or something else which treats a hash as a set.

[hdb]

There is probably no general solution to the problem but each application requires another way of doing the testing. For example, in your serialization example, you could do sorting on the outcome:

$result = join '&', sort split '&', $result;

just for your testing. This seeems to be simple enough that no new bugs are introduced and will not require a sort in your production code.

While de-serialization could be a solution as well, one has to be very careful as it adds additional complexity. For example, if your serialization function returns "a=b&c=d&a=b" because of some bug, it could easily be "fixed" by a de-serialization procedure:

my %hash = map { split '=', $_ } split '&', $result;

Clearly, there are ways around this, like testing for the length of the string as well. But one has to spend extra time for each application and additional complexity can not be avoided.

[vsespb]

Unless keys() can be switched to a test mode to produce predictable results (doesn't need to be sorted, just give me something predictable, like before perl 5.17), what are the options?

To test hash_serialize():

I would deserialize it to memory, and compare with original hash (with something based on "sort keys" or with cmp_deeply).

However this violates some ideas of unit testing (while it's ok for integration testing).
If it's worrying you - test it for 1-element hash and 2-elements hash (yes, all 2 permutations), and stop. Do other testing with deserialization.

To test other code, which uses hash_serialize():

mock (fake) hash_serialize().

[ikegami]

Unless keys() can be switched to a test mode to produce predictable results (doesn't need to be sorted, just give me something predictable, like before perl 5.17), what are the options?

It hasn't been predictable since 5.8.1. It just happened to be the same most of the time.

[saintmike]

I meant "reproduceable", not "predictable" here.

[ikegami]

Same thing. It hasn't been reproduceable since 5.8.1. It just happened to be the same most of the time.

[davido]

How many keys are there? I'm sure it's not always the same, but what's the most you're likely to see?

[saintmike]

At this point it's more of an academic exercise, I'm looking for a general solution to the problem of having a unpredictable function somewhere deeply embedded in my call hierarchy, and writing unit tests to confirm the result.

Do other languages have this problem, except for very few functions like random()?

[vsespb]

Another possibility for your particular case is to sort keys. Performance penalty is small, and having random URLs generated each time is a bigger problem and can cause some caching issue and other troubles.

[saintmike]

You don't know if the performance penalty is small, who said the function isn't going to be used to run a billion times a second?

The mere thought of dealing with this by adding useless production code to fix a test problem I find very disturbing.

[vsespb]

In this my answer I am talking not about fixing test, but about adjusting your business-requirements. I don't see case when having program output to be URLs with random parameters ordering is good idea. Better normalized URLs.

[DrHyde]

Unless keys() can be switched to a test mode to produce predictable results ...

Several hours before you wrote that, someone else told you what the relevant environment variables are.