Re: forking and dbi

[ehdonhon]

That's a lot of forking all at once, especially if you have several hundred objects you are polling. I have a feeling that you are exceeding the max connections to your database, and thus causing this code to execute:

        if (!$child_dbh) {
            exit(0);
        }

That would explain your 'handle destroyed without disconnect messages'. To test this theory, try putting a disconnect command right before that exit and see if you still get the same warnings.

My first suggestion would be that you look at Parallel::ForkManager. You can then tune your program to only run x processes at a time. It will make your program more scalable and keep it from getting out of control.

My second suggestion would be to try ditcing the $child_dbh all together. I have a feeling that all of the child processes should be able to share the same $parent_dbh. (I might be mistaken here, give it a try).

[djantzen]

I have a feeling that all of the child processes should be able to share the same $parent_dbh.

The problem is that when you fork, you duplicate the entire contents of the current process. However, sometimes a process has certain characteristics that are not pure data, but that are tied specifically to the process in which they were initiated. In the case of database handles, the issue is the socket connections established prior to the fork that uniquely relate the process that created them to the database itself. Those sockets can't be treated merely as pure data and hence cloned, because they constitute a relationship between two processes. And that relationship gets broken during the cloning procedure.

AFAIK the only way to fork and share a database handle is to use interprocess communication, or to have some kind of "smart" forking procedure that reexecutes particular pieces of the code that occur before the fork and which cannot be directly cloned. But that would be hard :^P

[mpeppler]

There are several issues with forking and using database handles opened in the parent.

At its most basic the underlying database client libraries must support this - the Sybase libraries do, but I have heard that the Oracle libraries do not.

Second, because the client usually maintains state associated with the connection you need to make sure that no operation is pending at the time of the fork. Otherwise you are likely to have a very confused connection.

Third, DBI maintains some state as well, and this has to be taken into consideration as well.

I know for a fact that I can fork a perl script that uses Sybase::CTlib and use the connection in the child. I can even fork it multiple times and use the same connection in multiple children, as long as the use is coordinated so that only one child has access to the connection at a time (see for example my Apache::Sybase::ConPool module.)
I have not tried this with DBI, but I would expect it to work, as long as no open statements exist at the time of the fork.

Michael