Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number

Using a single database handle or multiple handles

by krfoot (Initiate)
on Oct 11, 2012 at 12:19 UTC ( #998429=perlquestion: print w/replies, xml ) Need Help??
krfoot has asked for the wisdom of the Perl Monks concerning the following question:

I'm curious about opinions on the use of database handles and sub-routines with the DBI. Specifically, is it more efficient to connect once to a database and then pass the database handle as an argument to sub-routines or should I create the database handle within each sub-routine? Does it matter either way?
  • Comment on Using a single database handle or multiple handles

Replies are listed 'Best First'.
Re: Using a single database handle or multiple handles
by zwon (Abbot) on Oct 11, 2012 at 12:29 UTC
    Generally it is more efficient to connect once. Each time you're connecting to database you're spending some time and resources. On the other hand if your process sending one request to database in ten minutes, it may be better to establish a new connection for every request.
Re: Using a single database handle or multiple handles
by blue_cowdawg (Monsignor) on Oct 11, 2012 at 13:18 UTC
        Specifically, is it more efficient to connect once

    There is some "setup time" involved with connecting to a database so I'd recommend doing that only once. One of the techniques I use for this involves writing a module that acts as a singleton. Here is a simplified example of that:

    package Customer::EmployeeDB; use DBI; my $instance = undef; sub new { if($instance) { return $instance; } else { bless { dbh=> undef } ,"Customer::EmployeeDB"; $self->{dbh}=DBI->connect ("DBI:mysql:database=employee","user +","secret") or die $DBI->errstr; $instance=$self; return $self; } } sub prepare { my ($self,$shift)=@_; my $sth=$self->{dbi}->prepare($sql) or die $self->{dbi}->errstr; return $sth; } 1;

    An example of using that module:

    | handwaving here use Customer::EmployeeDB; my $db = new Customer::EmployeeDB(); my $sth = $db->prepare("select * from employee order by surname"); $sth->execute(); | do something with it

    As I said, that's an incredibly simplified version of the actual family of modules I crafted to do the work. I actually go one step further and create modules to encapsulate the queries against the database and those modules instantiate the module encapsulating the database.

    Using the singleton behavior ensures that only one connection to the database is made within the entire script. When I profiled the resultant code I was amazed at how much time it took to connect to the database (the real world example was against Sybase, so your mileage may vary) relative to where time was spent elsewhere. The code I had inherited actually made multiple connections to the database and the client wondered why it ran so poorly.

    Peter L. Berghold -- Unix Professional
    Peter -at- Berghold -dot- Net; AOL IM redcowdawg Yahoo IM: blue_cowdawg
      Connecting to a modern Sybase server can cost between 100 to 300ms, depending on the amount of stuff that each connection needs to set up (e.g. if there are a lot of roles associated with the login being used, then this will take additional time).

      It also puts a fairly heavy load on the server (we had systems with > 200k new connections per day which were showing very serious performance issues).

      So I would always strive to limit the number of connections that you create, both for speed, and for load on the server.


Re: Using a single database handle or multiple handles
by tobyink (Abbot) on Oct 11, 2012 at 13:45 UTC

    Depends on the database. For most, a single handle is preferable.

    Microsoft SQL Server (and presumably Sybase too), when connected to using the TDS libraries does not allow multiple concurrent queries to execute on the same handle. You need to finish one query before making another. Thus patterns where you do:

    my $results = $dbh->someQuery; while (my $row = $results->fetchrow_array) { $dbh->someOtherQuery; }

    will not work unless you open a second connection. (I think some newer versions of TDS do actually emulate concurrent queries by transparently opening a second connection when required.)

    perl -E'sub Monkey::do{say$_,for@_,do{($monkey=[caller(0)]->[3])=~s{::}{ }and$monkey}}"Monkey say"->Monkey::do'

      Newer MS SQL Server drivers (and some other commercial ones) have an attribute called MARS_Connection which when set to 1 enables Multiple Active Result Sets. However, it is generally slower and has a number of restrictions. I wrote a small article on it for DBD::ODBC at Multiple Active Statements (MAS) and DBD::ODBC.

Re: Using a single database handle or multiple handles
by shmem (Chancellor) on Oct 11, 2012 at 14:03 UTC
    Does it matter either way?

    Of course it matters, but that's application specific.

    What zwon says, and then:

    • If your subroutines running time is so long that the overhead of setting up the database connection is negligible, you would delegate setting up the handle to each subroutine.
    • If your subs comprise transactions and you need to close the connection after each, you would let the sub set up the connection.
    • If you have many small operations, scattered over subroutines each running for a short time, you would pass the handle into the subs.
    • If database stuff is merely an initializing task, or a side aspect of your application, you might want to encapsulate the database stuff into a subroutine.
    • In a single threaded architecture, you could use one handle. In a multithreaded environment, you'd create a handle in subroutines which comprise a thread.

    Then you might have long running tasks. Could there be time-outs? Does your application fork? and so on...

      If your subs comprise transactions and you need to close the connection after each, you would let the sub set up the connection.

      This seems a somewhat strange procedure, or construction. Transactions are committed or rolled back by telling the database to COMMIT or ROLLBACK (and dbi can do either), not by closing the connection...

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://998429]
Approved by davies
Front-paged by Corion
and !@monks...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (4)
As of 2018-06-19 22:35 GMT
Find Nodes?
    Voting Booth?
    Should cpanminus be part of the standard Perl release?

    Results (115 votes). Check out past polls.