Re: What is the better way to store and display data.

[aaron_baugher]

It really depends on the use. If I'm saving a list of items for my own use only, and the list isn't terribly long, and I'm not going to do complex queries on it, I'm likely to just save it in a text file, as you've shown it here. A text file gives me several advantages:

• It's quick to create. No need to set up a database or design tables with keys and so on; just type in the data with a consistent pattern. If it's already in this format, even better.
• It's fast and easy to edit. I can edit it with a simple text editor via SSH from anywhere. If I want to make a change to John Doe's entry, I can do that far faster with a text editor than I ever could through the most convenient GUI -- and I don't have to design the interface.
• It's trivial to move it to or duplicate it on another system, simply by copying the file. Yes, databases can be copied, but generally they should be dumped/loaded, which is at least twice as many steps.
• It's reasonably easy to parse it and pull out the data I want. In your sample, set the input record separator to a blank line and grep records for whatever. Simple.

On the other hand, if other people need to be able to work with the data, or if there are millions of records, or if I need to do complex queries like "WHERE city = 'this' AND Status LIKE '%that%', it's probably worth the time to put it into a real database.

The one thing I wouldn't be likely to do with data like this would be to put it in a hash. First of all, a hash requires unique keys. Can you be sure there will never be two John Does in your list? So you're probably looking at an array, not a hash. And you're going to need code to load the data into the array from its current format, and then print it back out of the array into this format. So why not use that code to save it in this clear text format, rather than saving the array in some serialized format? Saved as plain text, it's much easier to edit, as I mentioned above.

[bitingduck]

On the other hand, if other people need to be able to work with the data, or if there are millions of records, or if I need to do complex queries like "WHERE city = 'this' AND Status LIKE '%that%', it's probably worth the time to put it into a real database.

An in-between approach would be to use a simple CSV file, since it sounds like there aren't that many entries, but to access it as if it were a database using DBD::CSV. It's then readable and treatable like a text file, and can be easily imported into spreadsheets, but you don't have to roll your own searches through the CSV.

[Anonymous Monk]

I like this:
but;


#!/usr/bin/perl
    use strict;
    use warnings;
    use DBI;

    # Connect to the database, (the directory containing our csv file(
+s))

    my $dbh = DBI->connect("DBI:CSV:f_dir=.;csv_eol=\n;");

    # Associate our csv file with the table name 'prospects' and
    # manually declare names for each of the columns

    $dbh->{'csv_tables'}->{'prospects.csv'} = {
            'col_names' => ["name", "address", "floors", "donated", "c
+ontact"]
    };

    # Output the name and number of floors using our column names

    my $sth = $dbh->prepare("SELECT * FROM prospects.csv WHERE name LI
+KE 'G%'");
    $sth->execute();
    while (my $row = $sth->fetchrow_hashref) {
        print("name = ", $row->{'name'},
              ",   Number of floors = ", $row->{'floors'}, "\n");
    }
    $sth->finish();


This code is from "http://perlmeme.org/tutorials/parsing_csv.html". But running it gives this:


DBD::CSV::st execute failed: Error 2034 while reading file /media/Micr
+oSD/code/prospects.csv: EIF - Loose unescaped quote at /usr/local/sha
+re/perl/5.14.2/SQL/Statement.pm line 1055
 [for Statement "SELECT * FROM prospects.csv WHERE name LIKE 'G%'"] at
+ propects.pl line 20.
DBD::CSV::st fetchrow_hashref failed: Attempt to fetch row without a p
+receeding execute () call or from a non-SELECT statement [for Stateme
+nt "SELECT * FROM prospects.csv WHERE name LIKE 'G%'"] at propects.pl
+ line 21.

[Tux]

Try playing with different ways to connect. Your CSV does not comply to minimal CSV rules, so you'd have to loosen them or complain to the author/generator of the CSV file.

my $dbh = DBI->connect ("dbi:CSV:", undef, undef, {
    f_dir                  => ".",      # Not needed, is default
    f_ext                  => ".csv/r", # advisable
    f_encoding             => "utf-8",  # probably needed

    csv_eol                => "\n",     # NOT needed!

    csv_allow_loose_quotes => 1,        # You /might/ need this
    csv_allow_whitespace   => 1,        # OR this

    RaiseError             => 1,        # Always a good option!
    PrintError             => 1,        # Also a good choice
    });

Look into the Text::CSV_XS documentation for what other csv_ options are allowed.

FWIW, I will release a new DBD::CSV later today.

[Anonymous Monk]

Watch for leading spaces in your file.