Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

Re: selecting columns from a tab-separated-values file

by johngg (Abbot)
on Jan 21, 2013 at 22:58 UTC ( #1014518=note: print w/ replies, xml ) Need Help??


in reply to selecting columns from a tab-separated-values file

You might be better off tolerating the expense of loading your data into a database of some sort once, perhaps SQLite, and thereafter take advantage of the speed of the database which is optimised for data look-ups, using SQL queries to access your columns. DBI and DBD::SQLite would facilitate this.

I hope this is helpful.

Cheers,

JohnGG


Comment on Re: selecting columns from a tab-separated-values file
Re^2: selecting columns from a tab-separated-values file
by ibm1620 (Beadle) on Jan 22, 2013 at 04:15 UTC
    Interesting idea. Can SQLite handle that capacity? 10^9 rows and about 80GB of data total?
      give a go and you'll find out ;) or you can look here: http://sqlite.org/limits.html

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1014518]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (5)
As of 2014-09-22 01:09 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (176 votes), past polls