I'm not entirely clear on what you actually will be doing with the data after getting it loaded into memory, but there is another option: Copy the dataset from the on-disk SQLite database to an in-memory SQLite database using DBD::SQLite's
sqlite_backup_from_file method.
my $mem_dbh = DBI->connect('dbi:SQLite:dbname=:memory:');
$mem_dbh->sqlite_backup_from_file($db_file_name);
# process, process, process...
# Don't forget to copy the in-memory data back to disk if
# you've made any changes that should be persistent; if not,
# skip this step.
$mem_dbh->sqlite_backup_to_file($db_file_name);
It won't be as fast as using a hash for your in-memory manipulation, of course, but it will give you access to the full capabilities of SQLite if you find yourself needing to, e.g., query the data in ways that aren't easy/straightforward on a hash.
For comparison, I recently tried this technique out on a program that inserted a bunch of data into an SQLite database file, but was running inconveniently slowly. By copying the database into memory, inserting the data, and copying the database back to disk, the run time dropped from about seven and a half minutes to around one second. Quite a nice speedup, especially for such a trivial change to the source.
