Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change

Re^2: How to improve this data structure?

by karlgoethebier (Curate)
on May 21, 2013 at 18:30 UTC ( #1034577=note: print w/ replies, xml ) Need Help??

in reply to Re: How to improve this data structure?
in thread How to improve this data structure?

I assume you wanted to say something like:

Regards, Karl

«The Crux of the Biscuit is the Apostrophe»

Comment on Re^2: How to improve this data structure?
Re^3: How to improve this data structure?
by sundialsvc4 (Abbot) on May 21, 2013 at 19:55 UTC

    Yeah, more or less.

    “A million records” is a volume that is “vaguely interesting” to SQLite ... it will take a few minutes’s one-time cost to import the data (and might not require a program).   A couple minutes more to add some indexes.   From that point on, you can use any tool or combination of tools that has a DB-interface (including Perl of course ...) to get the job done, and now the query-engine is the one that’s doing all the heavy lifting.   So long as the “transactions” caveat is carefully adhered-to esp. when doing updates, it’s really quite a remarkable piece of software engineering.   (It’s rare when a piece of software genuinely surprises me blows me away.   Perl/CPAN did it.   So did this.)   I suspect that most of the things that the O.P. is right now “writing programs to do” can probably be reduced to a query (and perhaps a now-trivial program to digest the results).   Furthermore, a huge bonus is that you can put results into a different table, which of course is a self-describing data structure.   (A single SQLite file can contain any number of tables.)

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1034577]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (4)
As of 2014-10-25 03:07 GMT
Find Nodes?
    Voting Booth?

    For retirement, I am banking on:

    Results (141 votes), past polls