Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Re: selecting columns from a tab-separated-values file

by johngg (Canon)
on Jan 21, 2013 at 22:58 UTC ( [id://1014518]=note: print w/replies, xml ) Need Help??


in reply to selecting columns from a tab-separated-values file

You might be better off tolerating the expense of loading your data into a database of some sort once, perhaps SQLite, and thereafter take advantage of the speed of the database which is optimised for data look-ups, using SQL queries to access your columns. DBI and DBD::SQLite would facilitate this.

I hope this is helpful.

Cheers,

JohnGG

  • Comment on Re: selecting columns from a tab-separated-values file

Replies are listed 'Best First'.
Re^2: selecting columns from a tab-separated-values file
by ibm1620 (Hermit) on Jan 22, 2013 at 04:15 UTC
    Interesting idea. Can SQLite handle that capacity? 10^9 rows and about 80GB of data total?
      give a go and you'll find out ;) or you can look here: http://sqlite.org/limits.html

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1014518]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others lurking in the Monastery: (3)
As of 2024-04-19 01:03 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found