Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister

Re: Multi-thread database script (spawn)

by tye (Sage)
on Apr 24, 2007 at 05:19 UTC ( #611651=note: print w/replies, xml ) Need Help??

in reply to Multi-thread database script

Using Perl on Win32 I would avoid threads (Perl sucks at threads1) and avoid fork (fork is emulated by Win32 Perl and only somewhat successfully and this emulation also uses Perl threads, which suck, in case I didn't just mention that). (:

As ikegami notes, if you do use threads (via fork or otherwise), be sure to connect to the DB from the child. Perl threads create a new instance of the Perl interpretter and simple scalar values get copied fine but things with operating system magic attached (like file handles such as network connections to a database) can't be copied as easily. And the module isn't written to expect a single connection to be copied and used by two instances at once. So connecting and then forking or creating a new Perl thread will usually not work.

But because you are using both Perl and Win32, I'd instead spawn a DB client and feed tasks to it. That is, have either a separate Perl script or just a separate way of invoking the same Perl script and then invoke that in a child process and when an event happens, hand off the time-consuming task to it.

This can sometimes be handled by something as simple as:

use IO::Handle; open DBCHILD, "| $^X $0 -dbchild" or die ...; DBCHILD->autoflush( 1 ); # ... sub eventHandler { # ... print DBCHILD $eventTaskDesc, $/; }

Then the DB child just reads STDIN for things that it should do.

- tye        

1 The reason good threads are nice is that they are more efficient in memory than fork, share everything implicitly (which is also why they are not for the unprepared as preventing race conditions due to all this sharing requires a significant investment in tools and techniques), and share very efficiently. Perl threads, however, are less memory efficient than regular fork, share almost nothing implicitly, and don't even share efficiently.

Replies are listed 'Best First'.
Re^2: Multi-thread database script (spawn)
by fenLisesi (Priest) on Apr 24, 2007 at 07:14 UTC
    As far as I could tell from quick glances, there has been a lot of recent p5p activity by jdhedden and others around threads. Would people in the know care to discuss what kinds of improvement we may expect on the threads front with 5.10, please? This may also be a good opportunity for Parrot developers to update us on the state of Parrot threads and developments since Parrot, threads & fears for the future.. Cheers.

      The problems with Perl 5 threads are not due to the developers being incompetent nor for lack of effort. Any improvements to the Perl 5 threading model are going to be small improvements because the big problems are fundamental. You can't just take a monolithic system like Perl 5 and throw threads into the design as an after-thought. And Perl 5, unluckily, is an especially good example of a system that strongly resists the addition of good threading.

      At this point, I suspect that you'd often get better performance on Unix Perl if threading was implemented using fork with IPC used for sharing variables (especially if mmap were used). That is how fundamentally unlike threads the current Perl 5 threading model has become after years of trying and failing to make Perl work with a more thread-like model.

      - tye        

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://611651]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (5)
As of 2021-06-13 09:32 GMT
Find Nodes?
    Voting Booth?
    What does the "s" stand for in "perls"? (Whence perls)

    Results (54 votes). Check out past polls.