Re^3: How to set variable names using Text::CSV?

[Tux]

my $dbh = DBI->connect ("dbi:CSV:", undef, undef, {
    f_dir      => "DEMO.db:",
    f_ext      => ".csv/r",
    f_schema   => undef,
    PrintError => 1,
    RaiseError => 1,
    });

The disadvantage of this when the data set grows bigger is that all of your CSV data is kept in memory, and probably even twice. That, plus some meta-data will quickly gobble up most of your available valuable memory. If you want an intermediate database to make a move to something real later on, and still want portability, skip DBD::CSV and use DBD::SQLite instead. It is lightweight, fast and portable and supports a lot more than CSV.

[Marshall]

Thanks! for your suggestion of SQLite, I will see if I can get it on Windows! After I wrote my post, I generated the starting database, about 14K lines. This will grow in a very short time to about 50K lines. I know from experience and a lot of testing on other Perl applications that I won't start having trouble until memory footprint reaches > 1/2 GB. But that can happen quickly!

The basic thing is to get started using SQL for this stuff! I want to port the existing process first (which can be done with some "lite" tools), then on the "features to do list" are things that will require MUCH larger DB's. So, yes, portability of code is a factor if it is possible to do.