Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Re: randomising file order returned by File::Find

by GrandFather (Sage)
on Mar 01, 2011 at 21:46 UTC ( #890837=note: print w/ replies, xml ) Need Help??


in reply to randomising file order returned by File::Find

Reading between the lines a little, especially in some of you later replies, it looks to me like you are using a cluster of machines to do some heavy duty image processing and that the individual machines are fighting over claiming images to process. If that is the case then a better solution may be to consider one of the job scheduling modules such as Schedule::Load (disclaimer: I've not used the module, but the docs imply it does the right stuff).

If you want to roll your own task management a common technique to manage scheduling and locking is to use a database which provides transacted processing to mediate locking and task queue management.

For a build and automated test system I've put together I use a mysql database. The system processes about 10,000 tasks a month across a build/test farm comprising 6 machines using a mixture of Linux, Mac and Windows boxes. The locking can be done, but it is tricky and needs care to get right!

True laziness is hard work


Comment on Re: randomising file order returned by File::Find

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://890837]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (12)
As of 2015-07-31 10:14 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (276 votes), past polls