Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked

Re^2: use threads for dir tree walking really hurts (MFT?)

by Tanktalus (Canon)
on Sep 11, 2016 at 16:12 UTC ( #1171552=note: print w/replies, xml ) Need Help??

in reply to Re: use threads for dir tree walking really hurts (MFT?)
in thread use threads for dir tree walking really hurts

Just as a side note, multiple threads (up to a limit) definitely can access the filesystem faster than a single one. On unix, for example, I regularly do rm -rf by forking off (in perl) two sub-processes per directory for recursion (so you can get a lot of processes going at once). I doubt Windows is significantly different in this aspect - when the filesystem loads a given directory, it likely loads the full inode/sector and then hands back one item at a time, meanwhile another thread could request a different inode/sector and start acting on that. Meanwhile, any changes would be made in memory and sync'd out when the OS felt it either had to or had time to, so this can overall be much faster.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1171552]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (4)
As of 2021-09-17 20:05 GMT
Find Nodes?
    Voting Booth?

    No recent polls found