Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re: Re: Re: Sharing database handle

by d_i_r_t_y (Monk)
on Apr 23, 2002 at 08:40 UTC ( #161237=note: print w/replies, xml ) Need Help??


in reply to Re: Re: Sharing database handle
in thread Sharing database handle

i'm forking about 1 process every second.

continuously?! surely not?!!... perhaps you should post the code?

if divide and conquer is not applicable then i have one other suggestion:

  1. open a local UNIX socket/pipe
  2. fork a predefined set of children (say, 8), have each of the children open a DBI handle to mysql
  3. set up the parent process to loop over your insert data spooling it to the socket that the children are all listening to/blocking on; meanwhile each of children are blocking in the $server->accept() call waiting for the parent to send them data
  4. upon reading a chunk of data, enter the insert routine and loop back to the accept
  5. continue until parent runs out of data

an example of this approach (a pre-forking server, like apache) is in the perl cookback in the chapter on sockets or IPC, though i don't have the book in front of me at the moment, so can't give specific recipe.

but you're probably right; unless you're loading >1Gb of data (which does or does not need transformation prior to insert), it's probably fast enough. ;-)

d_i_r_t_y

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://161237]
help
Chatterbox?
[marto]: I doubt it even exists as a building site :P
[ambrus]: Corion: does building airports "somewhat outside of" a city ever work? cities always grow around the airport quickly, because it's practical to be close to it, and after that, people start to complain that the airplanes land so close to their house and sh
[ambrus]: ould keep down the noise

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (7)
As of 2017-02-23 12:04 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Before electricity was invented, what was the Electric Eel called?






    Results (346 votes). Check out past polls.