http://www.perlmonks.org?node_id=890837


in reply to randomising file order returned by File::Find

Reading between the lines a little, especially in some of you later replies, it looks to me like you are using a cluster of machines to do some heavy duty image processing and that the individual machines are fighting over claiming images to process. If that is the case then a better solution may be to consider one of the job scheduling modules such as Schedule::Load (disclaimer: I've not used the module, but the docs imply it does the right stuff).

If you want to roll your own task management a common technique to manage scheduling and locking is to use a database which provides transacted processing to mediate locking and task queue management.

For a build and automated test system I've put together I use a mysql database. The system processes about 10,000 tasks a month across a build/test farm comprising 6 machines using a mixture of Linux, Mac and Windows boxes. The locking can be done, but it is tricky and needs care to get right!

True laziness is hard work
  • Comment on Re: randomising file order returned by File::Find