Beefy Boxes and Bandwidth Generously Provided by pair Networks
Come for the quick hacks, stay for the epiphanies.
 
PerlMonks  

What do people want from DB abstraction modules?

by techcode (Hermit)
on Jan 05, 2006 at 14:34 UTC ( [id://521216]=perlquestion: print w/replies, xml ) Need Help??

techcode has asked for the wisdom of the Perl Monks concerning the following question:

There are modules that do this and that for you - but they are mostly written based on the authors idea of DB abstraction. I don't like the Class::DBI - not just for performance, but because of IMO strange way of working with it. I like better the approach of others like say SQL::Abstract ...

Some time ago - I started writing my own. And of course implemented stuff that I found useful yet boring to type as it's basically always the same. Sure there are modules that do some (or all) of the stuff my module does - but I'm doing it for practice anyway ;)

What I would like to know is - What do you people want from such a module?

One of the things that I found very useful in working with DB is Singleton pattern. It allows me to set the connection (or for several of them) data once and then simply ask for new instance where ever I need to work with DB - instead of "passing around the code" the initial DBH.

# Somewhere in init stage use DBIx::Handy; # need a better name ... DBIx::Handy->add_db(dsn => 'dbi:mysql:autoreorder', username => 'alex', name => 'oscommerce'); # Then latter my $DB = DBIx::Handy->get_instance(); my $SQL = "SELECT p.products_id, p.products_image, pd.products_name, pd.products_description FROM products p INNER JOIN products_description pd ON p.products_id = pd.products_id WHERE p.products_id = ?"; my $products_data = $DB->execute(sql => $SQL, data => [$form->{id}], database => 'oscommerce')->fetchrow +_hashref(); my $template = $self->load_tmpl('Product/details.dwt'); $template->param(%{$products_data}); return $template->output();

I didn't felt the need to add much SQL generation functions as I usually have either very simple or very complex queries (that I couldn't generate with abstraction modules). But it does generate SQL for simple INSERT and UPDATE statements.

It simply compares the table columns to keys in hash provided (data to be written) and only include those fields that are found in both.

So for instance you get hash (hashref) from CGI.pm, pass it througth Data::FormValidator and forward it to DBIx::Handy for update/insert.

Another thing that I added recently is possibility to return AoH reference from a query. I very often need those to print search results. And I just pass the reference to say HTML::Template.

my $schools = $DB->execute(sql =>'SELECT * FROM users ORDER BY us +ername', method =>'fetchall_aohref'); my $t = $self->load_tmpl('Admin/view_admin_users.dwt'); $t->param('schools' => $schools); return $t->output();

Currently everything is MySQL specific (might work on other DB's) but I'm looking for a way to support other RDBMS as well. Either through sub classing and overriding needed methods, or in a similar way DBI does it with all DBD::XYZ modules ...


Have you tried freelancing? Check out Scriptlance - I work there.

Replies are listed 'Best First'.
Re: What do people want from DB abstraction modules?
by exussum0 (Vicar) on Jan 05, 2006 at 17:01 UTC
    • Simple things should be made simple. In DBI, to get a row of data using a primary key for the bind variable, I need to create a statement handler, bind a parameter, execute, and return some structure. 4 lines of code I use everywhere, but I wind up keeping DBI libs so that I don't have to do that all the time. Pains me to see people write those 4 lines everywhere.
    • A consistent interface framework for CRUD'ing my data. SQL is nice, but frankly, it's the pits going from a 4 table query, into a set of objects. ORMs do that silliness nicely.
    • Requesting data also always requires SQL. Wouldn't it be nice if I can use a set of the ANSI SQL language and have it translated to my DB's version of SQL. I'm tired of converting sql statements from one db's SQL into another. Don't even start me on openning a connection for rollback-able transactions.
    • Consistency in the low-level operations. I don't trust people's driver level libraries to be remotely consistent. I want to setup, breakdown and have queries adhere to timeouts in a consistent fashion. Setting up evals with alarms is something people should not have to think about.
    • Cross DB support. Ever need to join two tables aross do different databases? It's the pits.
      In DBI, to get a row of data using a primary key for the bind variable, I need to create a statement handler, bind a parameter, execute, and return some structure. 4 lines of code
      Or one line: my @row = $dbh->selectrow_array($sql,{},$key);
      Cross DB support. Ever need to join two tables aross do different databases?
      DBD::AnyData can make that pretty simple for some queries:
      my $data = $dbh->selectrow_array( "SELECT $cols FROM IMPORT(?) JOIN IMPORT(?) ON(...", {}, $postgresql_sth, $mysql_sth );
        Or one line: my @row = $dbh->selectrow_array($sql,{},$key);
        Never noticed that before on the $dbh object. But more of those types of things are useful.
        my $data = $dbh->selectrow_array( "SELECT $cols FROM IMPORT(?) JOIN IMPORT(?) ON(...", {}, $postgresql_sth, $mysql_sth );
        I kinda like that. The funky IMPORT(?) doesn't seem standard, but something more "SELECT * FROM $dbh1 A, $dbh2 B where b.something = a.something" seems more natural. In some databases, I frequently do, "
        SELECT * FROM ( SELECT * FROM A WHERE A.something = 1 ) AA, ( SELECT * FROM B WHERE B.something = 2) BB WHERE AA.bob = BB.bob
        nitpick pick pick :)

        ----
        Give me strength for today.. I will not talk it away..
        Just for a moment.. It will burn through the clouds.. and shine down on me.

      Cross DB support. Ever need to join two tables aross do different databases? It's the pits.
      Could you describe what do you mean? My modules does support work with several databases (different dbh's) at the same time - and should support multiple DB types (MySql, SQLite, Postgree ...). But I have no idea how to possibly join two tables in different DB's (with JOIN or say sub query).

      Simple things should be made simple. In DBI, to get a row of data using a primary key for the bind variable, I need to create a statement handler, bind a parameter, execute, and return some structure. 4 lines of code I use everywhere, but I wind up keeping DBI libs so that I don't have to do that all the time. Pains me to see people write those 4 lines everywhere.
      I actualy never used bind variables. I don't like them for some reason - and find it more convenient to place everything inside hashes. Guess it's because most modules accept/give hashes in one way or another.

      But you could do something like:
      my ($var1, $var2, $var3) = $DB->execute('SELECT field1, field2, field3 + FROM whatever')->fetchrow_array();
      Or maybe even something like following.
      my ($var1, $var2, $var3) = $DB->do('SELECT field1, field2, field3 FROM + whatever WHERE id = 1');

      Requesting data also always requires SQL. Wouldn't it be nice if I can use a set of the ANSI SQL language and have it translated to my DB's version of SQL. I'm tired of converting sql statements from one db's SQL into another. Don't even start me on openning a connection for rollback-able transactions.
      Oh well this isn't perfect world. IMHO that's the job of RDBMS in the first place - they should stick to standards. Only thing I can do is support functions that will be there for as many DB types.

      Consistency in the low-level operations. I don't trust people's driver level libraries to be remotely consistent. I want to setup, breakdown and have queries adhere to timeouts in a consistent fashion. Setting up evals with alarms is something people should not have to think about.
      I'm not sure I follow you on this?


      Have you tried freelancing? Check out Scriptlance - I work there.
        Could you describe what do you mean? My modules does support work with several databases (different dbh's) at the same time - and should support multiple DB types (MySql, SQLite, Postgree ...). But I have no idea how to possibly join two tables in different DB's (with JOIN or say sub query).
        Sometimes one needs to work w/ two different database handles. I can get the data from both, and know that two columns, one from either database, can be used in a join. Only problem is, db1 and db2 know nothing about each other. Hell, they may not even be the same DB. I know the solution to the problem. Get all the data from db1 I need, get all the data from db2 I need. Iterate on one, joining results from the other using this one column.

        It doesn't come up often, but it does come up. The module would need to be fed two queries, and two joining column numbers to create a new set of results.

        Oh well this isn't perfect world. IMHO that's the job of RDBMS in the first place - they should stick to standards. Only thing I can do is support functions that will be there for as many DB types.
        They should but they don't. It doesn't make the problem go away.
        Consistency in the low-level operations. I don't trust people's driver level libraries to be remotely consistent. I want to setup, breakdown and have queries adhere to timeouts in a consistent fashion. Setting up evals with alarms is something people should not have to think about.
        The only operations I can think of that are common on DBI across multiple DB types is the username and password being setup. That's fine. I have no way of saying, "if a query being executed takes too long, abort and close the statement". DBI handles a lot of this, but not everything. There's an extra few metres left to cover.

        ----
        Give me strength for today.. I will not talk it away..
        Just for a moment.. It will burn through the clouds.. and shine down on me.

        Could you describe what do you mean? My modules does support work with several databases (different dbh's) at the same time - and should support multiple DB types (MySql, SQLite, Postgree ...). But I have no idea how to possibly join two tables in different DB's (with JOIN or say sub query).

        I think that's exactly what's being requested -- a join or sub query using multiple databases. Typically, you can rewrite a join as a subquery, or vise-versa, and databases typically handle two different styles of joins (sort-merge or correlated).

        With a sort-merge, you take the results from each table, sort them on the fields that it's getting joined by, and then work your way through the lists, finding entries in each list that have those fields in common.

        With a correlated query, you get the results from one table, then for each instance of the fields that you're trying to merge on, you query the other table.

        Depending on the size of the two tables being joined, one or the other may be more efficient. (and of course, with a correlated query, if it's an equijoin, you can work from either table first.) Database tuning involves storing histograms of the fields that you're going to be joining on, so you have a clue which way is going to be the most efficient.

        Oh -- and for those using Oracle on both ends -- look into what they call 'database links'.

Re: What do people want from DB abstraction modules?
by cees (Curate) on Jan 05, 2006 at 18:14 UTC

    In order of importance I want the following from a DB abstraction module:

    1. Make DB interactions safer
    2. Make DB interactions easier
    3. Make DB interactions consistent

    Using DBI is not that difficult, but it is very repetitive, and often very verbose. Repetitive and verbose tasks tend to suffer from lazy mistakes in the code (especially from cut and paste errors).

    By simplifying the access to the databas, I make my life easier, while also reducing the chance of programming errors (in my code, not necesarily the abstraction module itself), and my code ends up more consistent.

    Now whether you abstract the DB using functions to do the repetitive work, or you use objects to represent the data is just personal preference, as both can gain you the same benefits.

    Personally I like treating my tables as classes and rows as objects. This means I have used Class::DBI for quite some time, but lately I have been tempted to use Rose::DB::Object, which so far has seemed much cleaner and nicer to work with than Class::DBI. I guess time will tell which one I stick with, but Rose::DB::Object should definately be on the list of 'things to check out' for people that are looking into database abstraction.

Re: What do people want from DB abstraction modules?
by suaveant (Parson) on Jan 05, 2006 at 15:28 UTC
    Everything, duh! ;)

     

    Sorry... couldn't resist, tho its true. :)

                    - Ant
                    - Some of my best work - (1 2 3)

      Please describe everything. :)
      Have you tried freelancing? Check out Scriptlance - I work there.
        Well... everything... first of all it should be able to connect to any database and all functionality should be supported transparently. Functionality that isn't in some databases should be supplied by the module. It should be able to push dates and times at you in a consitent and configurable manner. You should be able to describe what you want in english and have it return data $dbe->plaintext_query("Give me all the clients from Schenectady, NY who are past due on their bill"). All functionality that is implemented should act the same, regardless of how the database handles the back end, count, group by, limit etc. Oh.. and it should make good coffee. :)

        It would do all this remarkably quickly.

        It would be called God::DB ;)

                        - Ant
                        - Some of my best work - (1 2 3)

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://521216]
Approved by Corion
Front-paged by kwaping
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others learning in the Monastery: (4)
As of 2024-12-07 15:06 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Which IDE have you been most impressed by?













    Results (50 votes). Check out past polls.