Database File Advice

[Anonymous Monk]

I have a project I am about to start that will require pulling information from text files and storing it in a database. I am searching for advice on what type of database file to use. My requirements are as follows:

* I need to create the databse programatically (no ODBC or OLE)
* I need the databse file format to be relatively small as I will be adding a lot of data.
* I need something that is easly searchable, updateable, and relatively fast.

What database format should I be looking at?

Thanks much in advance.

[jeffa]

How about SQLite? The corresponding CPAN module is DBD::SQLite. If you decide to upgrade to an RDMS, you should be able to use The SQL Fairy without a hitch (not to mention you shouldn't have to change most of code either).

Also, take some time to familiarize yourself with DBD::AnyData. If you play your cards right, this project will write itself. ;)

[liz]

Too bad that you need to download and install most of CPAN to get SQL::Translater / The SQL Fairy installed. ;-(

Liz

[Anonymous Monk]

And that's a problem how? You should have most of those modules anyway (I will never be able to understand how people perceive prerequisites which shave years of development time as problems).

[dragonchild]

You don't use a database file, you use a database and use SQL to search it. (At least, that's the standard.) You will probably use DBI and the appropriate DBD modules to access it from your scripts.

Most small projects use MySQL as their database, though PostgreSQL is popular. For large projects, you might have access to Oracle or Sybase, and there's many others. Tiny project might be able to get away with SQLite.

More info could help, like what system you're on. If you're on Windows, SQL*Server (accessed using [DBD::Sybase) is often a good choice.

[CountZero]

Any database using SQL would fit the description.

SQL allows you to define, create and delete your tables; it is ideally suited to search the database, insert, update and delete records and most databases servers (such as MySQL) are relatively fast.

The DBI-framework and the DBD-drivers allow easy interfacing with most database engines/servers and if you later want to change the underlying database, you can do so with little or no change to your programs (unless you have relied on database specific things -- which you should avoid if you can).

[merlyn]

Please stop suggesting MySQL for new starts. Read the MySQL gotchas about the failures.

If you want very fast and simple, use DBD::SQLite. If you want fast and full-featured, use PostgreSQL. MySQL just doesn't cut it any more. "MySQL is sooooo 90's".

[chromatic]

You know, that same site has PostgreSQL gotchas too.

If someone said "Perl is soooo 90s, use Python instead", you'd likely object. Rightfully so — I would too.

There are good reasons to use either database (and I've used both on different projects), but "it's soooo 90's" is not one of them. People who ask "which database should I use?" deserve better answers.

[CountZero]

Everyone's mileage varies.

I use MySQL a lot (indeed in "write once read many"-situations) and I'm quite happy with it, so I will keep suggesting it and you are free to tell people they should do something else.

[3dbc]

DBD::CSV has always worked when I just want to create a simplistic database format conducive for cross platform portability and the file format is relatively small since it contains only comma separated variables. As for the searching and updating I would recommend PERL DBD::CSV or you could use an abundance of products with a wide range of functionality like M$ Excel, M$ Access, Notepad, Edit Plus, even VI, etc. to view the data and search+update the data.

-3DBC