Re^3: Testing my tests

[LanX]

> We all do this, by default, don't we?

manually yes, not generated.

Szabgab only hinted about tests that handle code like $a+ switched to $a-

so in the following case

 
sub foo { 
   my ($a,$b) = @_;
   return $a + $b;
}

testing foo(*,0) (with * any number) will always pass, even if you change + to -

If several pairs of values were generated by a random generator and the result was approved manually, then the likelyhood of a false positive would be minimal.

And new test input could be generated quickly, depending on the manual decision if the code change was really intended or just a bug.

As I said to (semi-) automatize this one needed to parse already a lot of the functions body.

Anyway I don't want to elaborate more, without further explanation by the OP. (the requirements sound paradoxial)

[szabgab]

I am not sure why do you see the requirements paradoxical. You were the closest by far to what I was thinking and brought an excellent example. I've updated the original post with that and some further explanation.

And you are right this probably requires the full introspection of all the code.

[LanX]

> the requirements paradoxical.

because without further explanation it sounded like

• wanting tests that never fail
• or having test heuristics intelligent enough to make you unemployed.

at least now I'm sure what your up to.

Regarding your update, you can't avoid false positive tests.

But like I said you can reduce the likelyhood of them drastically for a reasonable prize by bundling many random test vectors.

Some my produce false positives, but hardly all of them.

In our case (2,0) may be OK but (666,42) would not.

[stevieb]

"(the requirements sound paradoxial)"

They do indeed, and with the respect I have for the OP, that's why I've been throwing out such odd concepts all day.

I don't know what exactly OP is looking for. Gabor didn't come back with further insight, so trial by current knowledge or at best peer-review is all we could throw forward.