Re: Re: Trojan Perl Distributions

[stvn]

A test using mock objects would not be a very good test.

Why do you say that? What really makes it different from testing with the real object? When DBD::SQLite writes to the database, is that testing Class::DBI or is it testing DBD::SQLite? Class::DBI really should not care what happens after it sends its SQL to the DBD driver, and to properly test Class::DBI, you should check that it generated the SQL you expected it to. By testing a SQLite database you are not really testing Class::DBI, but the side-effects of Class::DBI's interaction with DBD::SQLite.

So why not use something like DBD::Mock? It behaves pretty much like your standard DBD driver, and you can both feed it mock results and read back a history of SQL statements you have run. IMO this would be a much more effective (and less intrusive) test of Class::DBI's functionality. I have actually been using DBD::Mock to test our inhouse OO-Relational mapping tool and I would never go back to using a real DB to test. It has not only increased the quality of my tests, but it takes me less than half the time and code it did to test it with a real database. I recommend it highly.

[perrin]

You make a good point, and DBD::Mock is pretty cool. However, the difficulty is that DBD::Mock just does whatever you tell it to do.

When I was developing the code for Class::DBI's single-instance functionality, I found some tests that failed because there was a mistake in their assumptions about primary keys. If the test suite had been written to use DBD::Mock, it would have just returned whatever it was told to, and this bug would not have been found by the tests. In other words, the failure had to do with the interaction between multiple methods, not with the failure of a single method. It was only because the test did a real query on real data that had been created earlier in the test that I was able to see the problem.

[stvn]

I see what you are saying, but no matter what integration tests like your example are tricky to write. But keep in mind that while you do feed the results into DBD::Mock, there is nothing to prevent you from feeding in incorrect results, or results you know will mess things up. In fact, that is one of the things it is best at, simulating error conditions that would otherwise require you to make a mess out of your database.

In alot of cases, I like to develop against a real database, because in the end, that is the situation it needs to work in. It just makes sense to me to do it this way. After I am satisfied my module works, I have been converting the live DB test code to use DBD::Mock and adding tests at the end to also verify my SQL statements. The conversion from DBD::mysql to DBD::Mock is pretty minimal, it only requires that a seed my results, which is much easier than writing the code to make, manage and destroy test database.