Beefy Boxes and Bandwidth Generously Provided by pair Networks DiBona
"be consistent"
 
PerlMonks  

Re^3: Testing methodology, best practices and a pig in a hut.

by chromatic (Archbishop)
on Feb 27, 2008 at 19:06 UTC ( #670706=note: print w/ replies, xml ) Need Help??


in reply to Re^2: Testing methodology, best practices and a pig in a hut.
in thread Testing methodology, best practices and a pig in a hut.

Yet for some modules, the test suite is fully a third of the code produced.

It's a big chunk of Pugs, Rakudo, Parrot, and Perl 5 as well.

Now I'm all for better abstractions and reducing the amount of work to get the right results correctly (I did extract Test::Builder to make this possible for testing, after all), but complaining that a certain unnamed hypothetical test suite seems big to you is just as silly as complaining that you used string eval in a certain unnamed hypothetical production program.

The interesting question to me is "Why?" I agree that a blanket prohibition on one or the other is unhelpful just as I agree that a blanket prohibition on the use of symbolic references or even indirect object notation is unhelpful and likely wrong. That doesn't mean I want to throw a novice head-first into the whole debate over all of subtleties of community idioms and best principles when all he or she wants to do is know why the program doesn't work.


Comment on Re^3: Testing methodology, best practices and a pig in a hut.
Re^4: Testing methodology, best practices and a pig in a hut.
by BrowserUk (Pope) on Feb 27, 2008 at 19:28 UTC
    but complaining that a certain unnamed hypothetical test suite seems big to you is just as silly as complaining that you used string eval in a certain unnamed hypothetical production program.

    My point was that test suites constitute an important part of a production development, even though they may not run on production machines. They consitute a significant part of the development effort. But the prevelant idiom, the use of Test::*, applies different standards to the two sides of projects.

    1. String eval is the underlying basis of what the Test::* modules do.
    2. If you encountered the level of c&p coding that exist in most Test::* test suites, in production code, you would (rightly) be up in arms about it.
    3. If you encountered a (say) config file validation scheme in production code that did:
      warn 'Line 1 of config file invalid' if $line[ 0 ] ne '[Section 1]'; warn 'Line 2 of config file invalid' if $line[ 1 ] =~ m[^filename:(\S+ +)] and -e $1; warn ....

      For 400 lines, you'd probably question the methodology used.

    That doesn't mean I want to throw a novice head-first into the whole debate...

    How many novices have responded in this thread? Where else should such debate take place? Or perhaps you feel that no debate on this subject is called for?


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      String eval is the underlying basis of what the Test::* modules do.

      You're going to have to show some code to prove that.

      If you encountered the level of c&p coding that exist in most Test::* test suites, in production code, you would (rightly) be up in arms about it.

      Even if I thought you had looked at "most Test::* test suites", the existence of bad code does not prove that it's impossible to write good code. Remember, I extracted Test::Builder so that people could write their own Test::* modules in part to reduce duplication in their test suites. I've also argued for years that refactoring tests is just as important as refactoring production code.

      How many novices have responded in this thread?

      Your original post talked about people who were learning through doing and implied strongly that you think a contrarian position is important for people who haven't been knocked around enough to see the nuance in things.

      If that's not novices, then perhaps you could rephrase things to be more clear. (Several other people in this thread have asked "What does this have to do with the Test::* modules?")

      Or perhaps you feel that no debate on this subject is called for?

      I suspect that other people object to what you write not because you're wrong but because you can be so unpleasant about it.

        I suspect that other people object to what you write not because you're wrong but because you can be so unpleasant about it.

        Hm. I see the debate is ended. Contrast this question:

        Or perhaps you feel that no debate on this subject is called for?

        With this statement:

        I don't see the value in being contrarian solely for the sake of making the world less clear.

        And now think about pots and kettles.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
        You're going to have to show some code to prove that.

        From Test::Builder v0.78. The bit between big black lines looks an aweful lot like a string eval to me?

        sub cmp_ok { my($self, $got, $type, $expect, $name) = @_; # Treat overloaded objects as numbers if we're asked to do a # numeric comparison. my $unoverload = $numeric_cmps{$type} ? '_unoverload_num' : '_unoverload_str'; $self->$unoverload(\$got, \$expect); my $test; { local($@,$!,$SIG{__DIE__}); # isolate eval my $code = $self->_caller_context; # Yes, it has to look like this or 5.4.5 won't see the #line # directive. # Don't ask me, man, I just work here. ###################################################################### +########## $test = eval " $code" . "\$got $type \$expect;"; ###################################################################### +########## } local $Level = $Level + 1; my $ok = $self->ok($test, $name); unless( $ok ) { if( $type =~ /^(eq|==)$/ ) { $self->_is_diag($got, $type, $expect); } else { $self->_cmp_diag($got, $type, $expect); } } return $ok; }

        And cmp_ok() is used as the final step of

        1. is_eq
        2. is_num()
        3. _is_diag()
        4. isnt_eq()
        5. isnt_num()

        Same file as above. String eval between the thick marks:

        sub _regex_ok { my($self, $this, $regex, $cmp, $name) = @_; my $ok = 0; my $usable_regex = $self->maybe_regex($regex); unless (defined $usable_regex) { $ok = $self->ok( 0, $name ); $self->diag(" '$regex' doesn't look much like a regex to me +."); return $ok; } { my $test; my $code = $self->_caller_context; local($@, $!, $SIG{__DIE__}); # isolate eval # Yes, it has to look like this or 5.4.5 won't see the #line # directive. # Don't ask me, man, I just work here. ###################################################################### +###### $test = eval " $code" . q{$test = $this =~ /$usable_regex/ ? 1 : 0}; ###################################################################### +###### $test = !$test if $cmp eq '!~'; local $Level = $Level + 1; $ok = $self->ok( $test, $name ); } unless( $ok ) { $this = defined $this ? "'$this'" : 'undef'; my $match = $cmp eq '=~' ? "doesn't match" : "matches"; local $Level = $Level + 1; $self->diag(sprintf <<DIAGNOSTIC, $this, $match, $regex); %s %13s '%s' DIAGNOSTIC } return $ok; }

        And that function underlies the following api's:

        1. like()
        2. unlike()

        Proof enough?

        You want rude. This is what I really think of Test::*

        Reimplementing a small subset of perl's native functionality--string/numeric comparisons and regex--, using a large, slow, verbose and complex hierarchy of OO code modules, in order to compare scalar variables against constants, when at the final step you are just going to use string eval and let Perl do those comparisons anyway, just so that you can convert the boolean results into a stream of 'ok's and 'not ok's.

        And that, just so that you can tie up stdin & stderr in order to produce a set of useless statistic is, in a word, fascile(*).

        (*)facile: arrived at without due care or effort; lacking depth;


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://670706]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (6)
As of 2014-04-21 08:00 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (492 votes), past polls