Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options

Re: randomising file order returned by File::Find

by GrandFather (Sage)
on Mar 01, 2011 at 21:46 UTC ( #890837=note: print w/replies, xml ) Need Help??

in reply to randomising file order returned by File::Find

Reading between the lines a little, especially in some of you later replies, it looks to me like you are using a cluster of machines to do some heavy duty image processing and that the individual machines are fighting over claiming images to process. If that is the case then a better solution may be to consider one of the job scheduling modules such as Schedule::Load (disclaimer: I've not used the module, but the docs imply it does the right stuff).

If you want to roll your own task management a common technique to manage scheduling and locking is to use a database which provides transacted processing to mediate locking and task queue management.

For a build and automated test system I've put together I use a mysql database. The system processes about 10,000 tasks a month across a build/test farm comprising 6 machines using a mixture of Linux, Mac and Windows boxes. The locking can be done, but it is tricky and needs care to get right!

True laziness is hard work
  • Comment on Re: randomising file order returned by File::Find

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://890837]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (5)
As of 2018-06-18 03:51 GMT
Find Nodes?
    Voting Booth?
    Should cpanminus be part of the standard Perl release?

    Results (107 votes). Check out past polls.