Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Re: randomising file order returned by File::Find

by GrandFather (Cardinal)
on Mar 01, 2011 at 21:46 UTC ( #890837=note: print w/ replies, xml ) Need Help??


in reply to randomising file order returned by File::Find

Reading between the lines a little, especially in some of you later replies, it looks to me like you are using a cluster of machines to do some heavy duty image processing and that the individual machines are fighting over claiming images to process. If that is the case then a better solution may be to consider one of the job scheduling modules such as Schedule::Load (disclaimer: I've not used the module, but the docs imply it does the right stuff).

If you want to roll your own task management a common technique to manage scheduling and locking is to use a database which provides transacted processing to mediate locking and task queue management.

For a build and automated test system I've put together I use a mysql database. The system processes about 10,000 tasks a month across a build/test farm comprising 6 machines using a mixture of Linux, Mac and Windows boxes. The locking can be done, but it is tricky and needs care to get right!

True laziness is hard work


Comment on Re: randomising file order returned by File::Find

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://890837]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (13)
As of 2014-12-22 17:04 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (121 votes), past polls