Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Re^2: Hash lookups, Database lookups, and Scalability

by davido (Archbishop)
on Oct 31, 2004 at 17:53 UTC ( #404150=note: print w/ replies, xml ) Need Help??


in reply to Re: Hash lookups, Database lookups, and Scalability
in thread Hash lookups, Database lookups, and Scalability

I'm curious what you mean by "DBD::SQLite is not a sound database". I know it's a single file, lightweight, minimal-SQL set database, but to me something that isn't sound is buggy. Is that what you're saying? Just curious. (By the way, I just discovered a GPF-producing bug if I switch my code to use selectrow_array() with a pre-prepared statement on Win32, so maybe you've got a point. ;)

And yes, I fully expected the database solution to be much slower than the in-memory hash solution. I mentioned that in my post. I was attempting to quantify how much slower, and how sensitive the database version is to dataset-size.

Wrapping in transactions definately helps. Take the example of my building the database initially. With transactions, it takes the blink of an eye, but without, it takes about 45 seconds to build the database on my system. But on the query side, I'm specifically trying to engage in single queries, and didn't want to see transactions influencing the speed. However, I agree it would be interesting to rewrite the test in such a way that it could take advantage of transactions, and I may do that just to see what comes of it.

Anyway, it's kind of fun to tinker with this stuff. ...hope you agree, even it the database discussion is diverging a little from core-Perl topics.


Dave


Comment on Re^2: Hash lookups, Database lookups, and Scalability
Re^3: Hash lookups, Database lookups, and Scalability
by pg (Canon) on Oct 31, 2004 at 18:16 UTC
    "Anyway, it's kind of fun to tinker with this stuff. ...hope you agree, even it the database discussion is diverging a little from core-Perl topics. "

    I agree 100 percent. I did notice that recently we are having more and more database related discussions here, which is very possitive, at least to my standard. This indicates to me that more and more people are using Perl for an increased range of application areas, and most likely the average size of the applications is going upwards. This is definitely a positive news.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://404150]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (8)
As of 2014-08-28 09:22 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (259 votes), past polls