in reply to
Re^2: Design flat files database
in thread Design flat files database
You could search a few thousand bytes of directory data, even using linear search, in far less time than it would take you to access that data.
Sorry, whilst I'm no expert on *nix filesystems, I think you are wrong.
Well, I did
say a few thousand bytes of directory data
, which isn't going to apply to 100,000 files, 1 million files or 100 million files. I was mostly trying to move the OP away from the directory-per-digit option. If the IDs are used to cross-reference messages (as they are for messages here in the monastery), then a database, rather than flat files, is even more compelling. I don't know how messages are stored in the monastery, but I strongly suspect it is via a database, not in flat files. You wouldn't want to run monastic searches against unindexed flat files, but that would be relatively easy to implement (efficiently) in most databases.
I have recently been trying to nudge the OP in the direction of databases, and that's a nudge I see reflected in many of the responses.