Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number

Re: randomising file order returned by File::Find

by roboticus (Chancellor)
on Mar 01, 2011 at 11:44 UTC ( #890732=note: print w/replies, xml ) Need Help??

in reply to randomising file order returned by File::Find


How about having a single script dig through the directory tree and link the items it finds to 100 output directories. Then each of your 100 processing scripts can troll through their own input directory for items to process. That way you can avoid most of the locking problems. If you make your script that assigns items to queues, it could even watch the output directories and generate image links in the smallest directory available to balance the load (in case some process more quickly than others).


When your only tool is a hammer, all problems look like your thumb.

  • Comment on Re: randomising file order returned by File::Find

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://890732]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (5)
As of 2018-04-20 07:41 GMT
Find Nodes?
    Voting Booth?