http://www.perlmonks.org?node_id=635302


in reply to how to speed the mail process?

Definitely take the advice of profiling your code and also make sure to profile your SQL query. For example if you don't have an index ( preferably a partial index ) on the mark column... but there is one obvious performance improvement I can suggest. Move the $dbh->prepare() outside of the outer most while. Because the query isn't changing, there is no need to re-prepare it on every call.

Another option to consider, if you're using a database that support transactions, is to fork say 30 of these daemons and change your logic to be along the lines of (in pseudo-code):

  1. BEGIN TRANSACTION
  2. SELECT FOR UPDATE * FROM customers WHERE mark=0 LIMIT 1
  3. mark_the_flag_in_msgdb()
  4. send your message
  5. COMMIT TRANSACTION

This way you can be processing more at any given point....

But what you should really do is move to a messaging oriented system where the front end sends a message to something like Apache's ActiveMQ. The daemon would then pick up messages to be handled, send the message, write the data to the db, and move on to the next message...

Hope this helps!

Frank Wiles <frank@revsys.com>
www.revsys.com