Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris
 
PerlMonks  

Re^2: Performance Trap - Opening/Closing Files Inside a Loop

by tachyon (Chancellor)
on Dec 10, 2004 at 08:30 UTC ( #413781=note: print w/ replies, xml ) Need Help??


in reply to Re: Performance Trap - Opening/Closing Files Inside a Loop
in thread Performance Trap - Opening/Closing Files Inside a Loop

We found that ext3 with the 2.4.x Linux Kernel was reasonably happy with 10,000 files in a directory but obviously unhappy with 1 million. By reasonably happy I mean other bottlenecks dominated affairs. I would be interested if anyone has done a study on the relation of file numbers per dir vs access time for different file systems that it a little more precise.

cheers

tachyon


Comment on Re^2: Performance Trap - Opening/Closing Files Inside a Loop
Re^3: Performance Trap - Opening/Closing Files Inside a Loop
by meredith (Friar) on Dec 10, 2004 at 21:31 UTC

    Out of curiousity, did you test with the 'dir_index' feature flag set? It allows the filesystem to use hashed b-trees for lookups in large directories.

    mhoward - at - hattmoward.org

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://413781]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (9)
As of 2015-07-06 15:40 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (77 votes), past polls