Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical

Re^8: Splitting up a filesystem into 'bite sized' chunks

by Preceptor (Deacon)
on Jul 10, 2013 at 22:59 UTC ( #1043583=note: print w/replies, xml ) Need Help??

in reply to Re^7: Splitting up a filesystem into 'bite sized' chunks
in thread Splitting up a filesystem into 'bite sized' chunks

File::Find let's you specify "prune" which enables or disables traversal. If you know your last checkpoint is /mnt/myhome/stuff/junk you can pattern match your file path, and turn off traversal until you get a match. (you may have to roll up your checkpoint a little if the target has been deleted in the interim).

That'll - hopefully - give me a restartable find. Being able to "skip head" in future (and thus distribute processing) may require a first pass, and tracking multiple checkpoints.

Thinking about it, some sort of start/finish and some way of compensating for "drift". But one checkpoint every 100k files takes a huge list down to merely large. (but still doesn't help your first pass, unless you can take some wild guesses for initial checkpoints and do that same drift compensation.)

  • Comment on Re^8: Splitting up a filesystem into 'bite sized' chunks

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1043583]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (5)
As of 2017-11-22 04:40 GMT
Find Nodes?
    Voting Booth?
    In order to be able to say "I know Perl", you must have:

    Results (315 votes). Check out past polls.