Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

What is the better way to store and display data.

by Anonymous Monk
on Aug 21, 2012 at 14:04 UTC ( #988746=perlquestion: print w/ replies, xml ) Need Help??
Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hi there Monks!

I have a list of names and related information from about 50 people that I want to add to a database. I would like to have it in a hash rather than inserting peace by peace into a database table. I think that using a hash in Perl will be less work than setting up database tables queries etc. Could anyone point me into the right direction or suggest the best way of doing that? I also after storing it I will have to parse this data and display it in the same format I am showing it here. The data structure for all of them are the same as in:
Mary Anna , Left City Code: 40 X, Central Location First ID 454, Second ID 456 Status: MD at 01 floor John Doe , Right City Code: 88 Y, Right Location First ID 555, Second ID 321 Status: SI at 33 floor ...50 more of these

Thanks Monks!

Comment on What is the better way to store and display data.
Download Code
Re: What is the better way to store and display data.
by choroba (Abbot) on Aug 21, 2012 at 14:07 UTC
    What kind of queries are you planning to run? How often will your list change (i.e. adding new people, removing them, changing their details)? Is it possible for two different people to have the same name? Can the same person be mentioned more than once in the file?
    لսႽ ᥲᥒ⚪⟊Ⴙᘓᖇ Ꮅᘓᖇ⎱ Ⴙᥲ𝇋ƙᘓᖇ
      Nothing to complex, the list will not change much, unless someone drops out of the group. No one will have the same name and all the block for each person will be unique. Is just getting this into some sort of data file parse and display the data.
Re: What is the better way to store and display data.
by ig (Vicar) on Aug 21, 2012 at 14:19 UTC

    There are many ways to store data. Tables in an RDBMS is only one. You might find perl serialization helpful to learn about some alternatives that would allow you to store perl data structures without translating them into RDBMS tables. Which method will be best for you depends on many factors you haven't described, as indicated by choroba.

Re: What is the better way to store and display data.
by aaron_baugher (Deacon) on Aug 21, 2012 at 14:45 UTC

    It really depends on the use. If I'm saving a list of items for my own use only, and the list isn't terribly long, and I'm not going to do complex queries on it, I'm likely to just save it in a text file, as you've shown it here. A text file gives me several advantages:

    • It's quick to create. No need to set up a database or design tables with keys and so on; just type in the data with a consistent pattern. If it's already in this format, even better.
    • It's fast and easy to edit. I can edit it with a simple text editor via SSH from anywhere. If I want to make a change to John Doe's entry, I can do that far faster with a text editor than I ever could through the most convenient GUI -- and I don't have to design the interface.
    • It's trivial to move it to or duplicate it on another system, simply by copying the file. Yes, databases can be copied, but generally they should be dumped/loaded, which is at least twice as many steps.
    • It's reasonably easy to parse it and pull out the data I want. In your sample, set the input record separator to a blank line and grep records for whatever. Simple.

    On the other hand, if other people need to be able to work with the data, or if there are millions of records, or if I need to do complex queries like "WHERE city = 'this' AND Status LIKE '%that%', it's probably worth the time to put it into a real database.

    The one thing I wouldn't be likely to do with data like this would be to put it in a hash. First of all, a hash requires unique keys. Can you be sure there will never be two John Does in your list? So you're probably looking at an array, not a hash. And you're going to need code to load the data into the array from its current format, and then print it back out of the array into this format. So why not use that code to save it in this clear text format, rather than saving the array in some serialized format? Saved as plain text, it's much easier to edit, as I mentioned above.

    Aaron B.
    Available for small or large Perl jobs; see my home node.

      On the other hand, if other people need to be able to work with the data, or if there are millions of records, or if I need to do complex queries like "WHERE city = 'this' AND Status LIKE '%that%', it's probably worth the time to put it into a real database.

      An in-between approach would be to use a simple CSV file, since it sounds like there aren't that many entries, but to access it as if it were a database using DBD::CSV. It's then readable and treatable like a text file, and can be easily imported into spreadsheets, but you don't have to roll your own searches through the CSV.

        I like this:
        but;
        #!/usr/bin/perl use strict; use warnings; use DBI; # Connect to the database, (the directory containing our csv file( +s)) my $dbh = DBI->connect("DBI:CSV:f_dir=.;csv_eol=\n;"); # Associate our csv file with the table name 'prospects' and # manually declare names for each of the columns $dbh->{'csv_tables'}->{'prospects.csv'} = { 'col_names' => ["name", "address", "floors", "donated", "c +ontact"] }; # Output the name and number of floors using our column names my $sth = $dbh->prepare("SELECT * FROM prospects.csv WHERE name LI +KE 'G%'"); $sth->execute(); while (my $row = $sth->fetchrow_hashref) { print("name = ", $row->{'name'}, ", Number of floors = ", $row->{'floors'}, "\n"); } $sth->finish();

        This code is from "http://perlmeme.org/tutorials/parsing_csv.html". But running it gives this:
        DBD::CSV::st execute failed: Error 2034 while reading file /media/Micr +oSD/code/prospects.csv: EIF - Loose unescaped quote at /usr/local/sha +re/perl/5.14.2/SQL/Statement.pm line 1055 [for Statement "SELECT * FROM prospects.csv WHERE name LIKE 'G%'"] at + propects.pl line 20. DBD::CSV::st fetchrow_hashref failed: Attempt to fetch row without a p +receeding execute () call or from a non-SELECT statement [for Stateme +nt "SELECT * FROM prospects.csv WHERE name LIKE 'G%'"] at propects.pl + line 21.
Re: What is the better way to store and display data.
by Anonymous Monk on Aug 21, 2012 at 17:07 UTC
    I would use a database for what it's meant for.
Re: What is the better way to store and display data.
by lidden (Deacon) on Aug 21, 2012 at 17:17 UTC
    With only 50ish people Storable should work well. A DB will scale much better if the list grows though. Storable is a core module.
Re: What is the better way to store and display data.
by SerZKO (Beadle) on Aug 21, 2012 at 18:52 UTC
    Hej Anonymous Monk, I agree 100% with aaron_baugher. If it is only 50 of those then there is no reason to use anything else then text files. And it's really not a problem if you have a file formated like this
    Mary,Anna,Left,40 X,Central Location,454,456,MD at 01 floor John,Doe,Right,88 Y,Right Location,555,321,SI at 33 floor Kalle,Anka,Middle,77 Z,Left Location,777,212,AT at 22 floor
    to get a printout like you described.

    I'm pretty sure that you can find a lot of examples on the web.

    If you are absolute perl beginner, try reading Robert's perl tutorial, otherwise, well you know. Happy Perl-ing

      Thank you all for the help, going to build based on the DBI:CSV code above, I like the flexibility on renaming columns. Thas right, its just around 50 records, a text based system should be just fine.
Re: What is the better way to store and display data.
by zentara (Archbishop) on Aug 22, 2012 at 09:29 UTC

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://988746]
Approved by Perlbotics
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (5)
As of 2014-07-10 01:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    When choosing user names for websites, I prefer to use:








    Results (198 votes), past polls