Beefy Boxes and Bandwidth Generously Provided by pair Networks RobOMonk
Perl-Sensitive Sunglasses
 
PerlMonks  

Fork-safe DBI handles via ChildHandles and InactiveDestroy

by etcshadow (Priest)
on Jun 07, 2007 at 03:30 UTC ( #619722=perlquestion: print w/ replies, xml ) Need Help??
etcshadow has asked for the wisdom of the Perl Monks concerning the following question:

Hey, all.

The setup: I'm working on some code that involves forking in a process with potentially several open DBI database handles (dbhs) to Oracle database handles. Oracle dbhs will encounter problems when the child processes terminate unless they have the InactiveDestroy attribute set on them in the child processes. Otherwise, essentially, when the child process terminates, during its global destruction phase, every dbh reports back to its respective database that the connection has terminated, which means that Oracle will not accept any further activity on that connection (from other processes). Even if that connection was never used in the child process, the mere act of the child process coming into existence and then exiting will foul the parent process's database handle... unless of course this InactiveDestroy attribute is set. The whole purpose of the InactiveDestroy attribute is, basically, to make the DESTROY on the dbh "inactive", i.e. it doesn't send a message to Oracle to the effect of "I'm dead, ignore any future communications from me".

So, anyway, because I may be performing a fork deep down inside of a function call, I may not be able to easily pull a list of all the dbhs that are hanging around in the process, waiting to get passively disrupted by the fork. At least, I hadn't been able to do so in the past. However, DBI has somewhat recently (I don't know when exactly) introduced an attribute called ChildHandles which returns an array of weak-referrences to all the child handles of a given handle object. This allows me to something like this:

sub MakeAllDBHsForkSafe { my %drivers = DBI->installed_drivers; foreach my $drh (values %drivers) { map { $_->{InactiveDestroy} = 1 } @{$_->{ChildHandles}}; } }
Which, if called as the first thing in the child process, will prevent the child's global destruction from fouling all of the dbhs in the parent. This seems like a complete answer to my problems.

The real question though, is if this is a safe action to take. The DBI docs on the ChildHandles attributes state explicitly that "The referenced array returned should be treated as read-only." I'm a little concerned (perhaps overly concerned) over the specific semantics of this. Does it mean that the array, itself should not be altered (like adding or removing elements to the array), or does it mean that the array contents should not be altered? Is anybody here familiar enough with the DBI internals to know? Am I just being paranoid for even asking? :-)

I have tested this, and it seems to work fine, but I don't know if there are any gremlins hiding in their that will bite me over this. It seemed like a good idea to at least ask.

Thanks in advance.

------------ :Wq Not an editor command: Wq

Comment on Fork-safe DBI handles via ChildHandles and InactiveDestroy
Select or Download Code
Re: Fork-safe DBI handles via ChildHandles and InactiveDestroy
by Zaxo (Archbishop) on Jun 07, 2007 at 03:59 UTC

    It's the safest thing you can do without completely redesigning your code. The DBI and Oracle maintainers will appreciate hearing of any problems you have.

    I always design to avoid conflicts like that, but old mistakes must be worked around.

    After Compline,
    Zaxo

      Re: designing around it: true. Unfortunately, this code was originally for perl 5.5 (we only recently upgraded to 5.8), which didn't support weak referrences. Holding some sort of internal list of all the connected dbhs without weak referrences would have meant (indirectly) subverting the implicit disconnect when dbhs fall out of scope (which is generally a good thing). That would have been a bad thing.

      As far as the fact that a fork can sort of corrupt various resources which aren't even used in the subprocess... well, that's unfortunate, but not isolated to DBI connections. A similar thing happens with IO buffers, necessitating that you do something like "flush all handles" before a fork. Unfortunately, this, too (at least used to be... someone may have fixed this since I last looked into it) has no good solution, as it requires the forking code to somehow reach out and grab a list of all of a process's resources of a type (in this case database handles, in that case buffered IO handles). Similarly, in apache, there's a necessary step of calling "cleanup_for_exec", so that child processes' sockets don't get selected for handling new requests. The list goes on, I'm sure... which is sad, because it means there's a whole laundry list of things you need to remember to do if you want to fork... forgetting to do any of them meaning a potential obscure bug.

      ------------ :Wq Not an editor command: Wq
        As far as the fact that a fork can sort of corrupt various resources which aren't even used in the subprocess... well, that's unfortunate, but not isolated to DBI connections.

        Exactly. To be really careful, a program which forks should avoid letting the kids inherit such connections.

        After Compline,
        Zaxo

        A few months ago, to make a point about some of the pitfalls of fork and IO, I wrote a few tests to show the potential problems (from a Un*x perspective following the R. Stevens books for instance), and all my tests were negative in the sense that (at least on cygwin -- I think I tested HP-UX which has been my perl platform for years, but I cannot remember for sure) implicit flushing was going on ...It would be interesting to know how much perl departs from Un*x on this (and since when).

        In general for this kind of problem, maybe the best way to go is a process manager maintaining a pool of processes each one with one DB connection, and some protocol to reach the process manager. Anyway I find the "forks" module interesting.

        cheers --stephan
Re: Fork-safe DBI handles via ChildHandles and InactiveDestroy
by kyle (Abbot) on Jun 07, 2007 at 11:08 UTC

    It's not clear from your writeup, so I'll mention it anyway. After the fork, even if you set InactiveDestroy, it's important that the child not use the database handle. Basically, after setting InactiveDestroy, you should then destroy it. I wrote about this in DBI, fork, and clone..

    As to your question, my reading of the documentation is that the array itself should not be modified. That is, you should not try to add a handle to it or splice one out or anything. What you're doing should be fine.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://619722]
Approved by ww
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (4)
As of 2014-04-18 02:06 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (460 votes), past polls