Beefy Boxes and Bandwidth Generously Provided by pair Networks Cowboy Neal with Hat
The stupid question is the question not asked
 
PerlMonks  

Re^2: How to improve this data structure?

by karlgoethebier (Deacon)
on May 21, 2013 at 18:30 UTC ( #1034577=note: print w/ replies, xml ) Need Help??


in reply to Re: How to improve this data structure?
in thread How to improve this data structure?

I assume you wanted to say something like:

Regards, Karl

«The Crux of the Biscuit is the Apostrophe»


Comment on Re^2: How to improve this data structure?
Re^3: How to improve this data structure?
by sundialsvc4 (Monsignor) on May 21, 2013 at 19:55 UTC

    Yeah, more or less.

    “A million records” is a volume that is “vaguely interesting” to SQLite ... it will take a few minutes’s one-time cost to import the data (and might not require a program).   A couple minutes more to add some indexes.   From that point on, you can use any tool or combination of tools that has a DB-interface (including Perl of course ...) to get the job done, and now the query-engine is the one that’s doing all the heavy lifting.   So long as the “transactions” caveat is carefully adhered-to esp. when doing updates, it’s really quite a remarkable piece of software engineering.   (It’s rare when a piece of software genuinely surprises me blows me away.   Perl/CPAN did it.   So did this.)   I suspect that most of the things that the O.P. is right now “writing programs to do” can probably be reduced to a query (and perhaps a now-trivial program to digest the results).   Furthermore, a huge bonus is that you can put results into a different table, which of course is a self-describing data structure.   (A single SQLite file can contain any number of tables.)

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1034577]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (14)
As of 2014-04-17 11:45 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (446 votes), past polls