Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Re: When is a flat file DB not enough?

by lhoward (Vicar)
on Jul 03, 2001 at 19:24 UTC ( #93540=note: print w/replies, xml ) Need Help??


in reply to When is a flat file DB not enough?

It is a generally accepted fact that no flat-file database can support more than 5,173 items until performance starts to deteriorate rapidly. The maximum number of elements that any flat file database can support is limited to 23,987.


Just kidding. The real answer is: it depends

  • How large are the items in the DB
  • In what ways are they accessed (read, written, updated)
  • How is the flat-file DB implemented ("by hand", DBD::CSV, or something else)
  • Are there any locking or atomicity issues to consider

Strictly speaking, the amout of data that a flat-file DB can handle is only limited by the maximum filesize that your filesystem will support. However, at some point performance will become unacceptable. About the only way for you to come up with a good answer is to benchmark your system. Loading it with various amounts of data and benchmarking performance of various sample operations.

Replies are listed 'Best First'.
Re: Re: When is a flat file DB not enough?
by TrinityInfinity (Scribe) on Jul 03, 2001 at 19:30 UTC
    I figured the filesize would be the key. This is all running on a solaris setup that handles files far bigger than mine, but I think it can handle a couple hundred items at best before the search and editing times start to get noticible. Thanks for the response! =)
      Don't think solaris is so easy with large files. Sure it can handle them, but the I/O bandwith may bite you!!!

      Linux is better with that kinda thing...

      Sinister greetings.
      perldoc -q $_

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://93540]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (5)
As of 2019-08-22 08:01 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?