Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Re: Queues, not the usual problem

by lhoward (Vicar)
on Jul 09, 2001 at 17:26 UTC ( #94967=note: print w/replies, xml ) Need Help??


in reply to Queues, not the usual problem

As arhuman mentioned above, MySQL may not be ideal because of its locking mechanism. However, a slightly different architecture (used by many queueing systems) could work well for you (with MySQL).
  • process(es) that insert into the queue - generators
  • quque manager process
  • process(es) that remove from the queue - consumers
With a single queue manager that is responsible for assigning queue ID's to new queue entries, and giving queue entries to the consumer processes you avoid the MySQL lock problem. You could use MySQL (or even flat files) to backup the queue manager in case it failed or you have too much data to fit in ram at once, or the queue manger could just direct and queue the data directly between processes.

Replies are listed 'Best First'.
Re: Re: Queues, not the usual problem
by rob_smith01 (Acolyte) on Jul 09, 2001 at 17:48 UTC
    Ive decided to write a script and spool the data instead due to dead lock. Its bound to happen with the amount of data Im using and processing.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://94967]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (4)
As of 2019-08-23 16:21 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?