DBIx::Simple - Your opinions/review please

[Juerd]

Fellow Monks,

if you spend a lot of time working on something, after a while things get blurry. DBIx::Simple recently got a lot of it re-coded, it was refactored and some features are deprecated. All user suggestions that I could track have in some way been dealt with. This resulted in a lot of "It's already there, if you look for it" and some "Sorry, I don't think that belongs in a *Simple* module", but also a bunch of new features. As said, things get blurry.

Perhaps something very important is missing, but just like my brain learned to ignore the mess in my office, I no longer see the code of DBIx::Simple clearly.

Last time, I waited a long time before releasing the new version. But that didn't really work well. DBIx::Simple is used in a lot of production environments, and I'd like to release bug fixes soon, and would like to introduce new simplifying features more often. That way, maybe people won't have to hack it and mess with its internals. (Although the recently added dbh and attr methods should avoid having to hack.)

One thing is missing that I know of. A test suite. To be honest, testing is one of my weaknesses. I know that I like having tests to rely on, and that I don't mind adding tests when I add new features. But I hate writing a test suite from scratch, especially for a module that I've been using for over two years already. Fortunately, someone volunteered to write a complete and thorough test suite for this module. Because of the circumstances, this has been delayed a few times already, but eventually, DBIx::Simple will have tests. So please don't complain about that.

As some readers will know, I'd like to know how many people use DBIx::Simple, or on how many boxes it is installed, or even how often it is downloaded. I'm not yet going to automate that, because I'd first like to know how people use DBIx::Simple.

I'm not asking you if you use it privately or commercially, I don't care if it's for an application that one person will use or for one that thousands rely on. (But if you want to tell me, please go ahead.) Things I'm very interested in right now are:

• Do you use it for throw away scripts, in production, both?
• How many queries happen per second? How large are the results (in rows, kilobytes)? (millions of tiny queries, tens of larger ones, etc)
• Do you use any of the currently deprecated features?
• What DBDs do you use it with?
• Do you use the "(??)"-thing? What's your opinion on having more like these, for example for easier use with INSERT and UPDATE?

Another thing I'm interested in, is what you think about DBIx::Simple in general. Is it fast enough? Does it focus on speed too much? Should it have less or more features? Is it just about right? How is the documentation? Do you hate or love its source? Is the Changes file detailed enough? Should I not be using EU::MM?

In case you haven't used DBIx::Simple yet, please do give it a try and let me know how things went. I'd really appreciate a thorough review, especially from someone who has experience with DBI and/or DBIx modules.

I might not like your suggestions. But I will read what you have to say and think about it.

Thanks in advance for helping me develop DBIx::Simple further.

<shameless>I wouldn't mind some more ratings =)</shameless>

[simonm]

Juerd -- Nice work.

A few random suggestions:

• If the subquery emulation has been deprecated, change the short tagline in the NAME section ("Easy-to-use...") to reflect only the primary purposes.
• The method named _die is confusing, since it doesn't have anything to do with the core die(); perhaps naming it something like done, finished, or destroy would make it clearer that this is a pre-destruction cleanup method.
• While I think the "keep_statements" functionality is nicely implemented, if its purpose is just for performance, might you be better off using the DBI's built-in prepare_cached instead?
• In addition to referring people to DBIx::Abstract, you might want to also mention SQL::Abstract, perhaps even with a short example showing how easy it is to use SQL::Abstract with your module.
• With regard to testing, I've struggled with the same issue; this kind of testing is trickier than for self-contained modules that don't have to talk to a database. My suggestion would be to use EU::MM's prompt() function to ask the user for a test DSN when Makefile.PL is run; note that EU::MM will skip this automatically when running from within CPAN.

Update: Fixed links, thanks Juerd.

[Juerd]

If the subquery emulation has been deprecated, change the short tagline in the NAME section ("Easy-to-use...") to reflect only the primary purposes.

You're right. It has already been removed, even :) I used to be proud of this feature, because it enabled users of MySQL to use subqueries and a lot of people seemed to really want that. But MySQL now supports that natively and thus of course in a much better way. And I don't want to maintain the ugly code needed for it just for database drivers that aren't used much anyway.

The method named _die is confusing, since it doesn't have anything to do with the core die(); perhaps naming it something like done, finished, or destroy would make it clearer that this is a pre-destruction cleanup method

Or during-destruction, or triggering-destruction. It's kind of hard to explain and at the time I really had trouble finding a good name. I still find that hard. Nothing describes it really well. I think _die is as bad a name as the other suggestions. For method names, I never really care about clashing with builtins.

While I think the "keep_statements" functionality is nicely implemented, if its purpose is just for performance, might you be better off using the DBI's built-in prepare_cached instead?

prepare_cached Is dangerous, as described in the documentation of DBI under the "Caveat Emptor" paragraph. Since DBI 1.40, there's a way to make it safe, but I use DBIx::Simple on boxes that have older versions of DBI. Its functionality is also driver dependent.

Another thing is that DBI doesn't limit the number of cached handles, which can be a serious problem in persistent environments like mod_perl.

Oh, and I need to control what happens to STHs because of the extensive error reporting. DBI could give the STH to something that isn't DBIx::Simple. (Which by the way is also a security implication, since you can use it without re-execute()ing. (Of course, nothing realli is safe in Perl, since even lexicals can be accessed from elsewhere...))

In addition to referring people to DBIx::Abstract, you might want to also mention SQL::Abstract, perhaps even with a short example showing how easy it is to use SQL::Abstract with your module.

Good idea. Will do! (By the way - the link in your post is broken)

Update: In fact, the more I think about it, SQL::Abstract is VERY compatible with DBIx::Simple: its return values can be used directly, without modification, with DBIx::Simple's query method. Although I don't like abstracting SQL (unless completely, that is: with Class::DBI), it's the most often requested feature. I could even wrap its four main methods entirely, as none of its methods clash with DBIx::Simple's methods.

That would probably something like: (untested)

sub abstract : lvalue { shift->{abstract} }

for my $method (qw/select insert update delete/) {
    *$method = sub {
        require SQL::Abstract;
        my $self = shift;
        $self->{abstract} ||= SQL::Abstract->new;
        return $self->query($self->{abstract}->$method(@_));
    }
}

Is there any reason I should or should not do this?

Another update: It's implemented in my local copy now, in an even simpler way. Just to make sure some things, I've emailed the author of SQL::Abstract with some questions, but I don't expect anything negative back. So probably DBIx::Simple 1.23 will have these four new methods.

With regard to testing, I've struggled with the same issue; this kind of testing is trickier than for self-contained modules that don't have to talk to a database. My suggestion would be to use EU::MM's prompt() function to ask the user for a test DSN when Makefile.PL is run; note that EU::MM will skip this automatically when running from within CPAN.

Someone else is working on the tests. I don't know what it'll look like :)

[hv]

I used to be proud of [the subquery emulation feature], because it enabled users of MySQL to use subqueries and a lot of people seemed to really want that. But MySQL now supports that natively and thus of course in a much better way. And I don't want to maintain the ugly code needed for it just for database drivers that aren't used much anyway.

Just for another viewpoint, I've not used DBIx::Simple, but your original post prompted me to skim the docs. The only thing there that seemed of possible interest to me was this feature - my work application targets MySQL v3, and while the next generation of the application will target a later version it is unlikely that the current generation will be re-targeted.

I've occasionally been hurt by the lack of subselects, and considered emulating them; discovering that someone has already done that was a pleasant surprise.

If it works, I'd suggest keeping it - don't expect MySQL v3 to disappear from production servers any time soon. (In fact, a quick check on http://www.mysql.com suggests that the subselects are in v4.1, while latest stable release is still at 4.0.20.)

Hugo

[Juerd]

Revision history for Perl extension DBIx::Simple.
                                                                                          
Incompatible changes are marked with "!!".
                                                                                          
1.23  Thu Jun 10 0:00 2004
        - New: DBIx::Simple::SQE.
        - New: $db->abstract.
        - New: $db->select.
        - New: $db->insert.
        - New: $db->update.
        - New: $db->delete.
        - These new methods use Nathan Wiger's SQL::Abstract (loaded on demand)
          and are based on a suggestion by simonm (perlmonks.org).
        - Yes, I lied. I said that this module would never get SQL abstraction,
          but this was too easy to add and doesn't hurt anyone who doesn't use
          it.
        - New: $db->begin, an alias for $db->begin_work.
     !! - Removed: $db->emulate_subqueries, $db->esq. (But see
          DBIx::Simple::SQE.)
     !! - Removed: $db->omniholder. (Now a constant: always "(??)".)

DBIx::Simple