Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Least impact DB for CPAN modules

by hsmyers (Canon)
on Oct 02, 2010 at 19:13 UTC ( #863099=perlquestion: print w/replies, xml ) Need Help??
hsmyers has asked for the wisdom of the Perl Monks concerning the following question:

I'm at the beginning of refreshing/cleaning one of my CPAN modules Chess::PGN::EPD which among other things requires a light weight database for some of it's functions. At the moment I'm using DB_File, but I'm completely open to other possibilities— mostly what I want is to impact my users as little as possible yet maintain functionality.

As a picture of the amount of information involved (not much by today's standards) here is what shows in a dos box:

C:\>dir Volume in drive C has no label. Volume Serial Number is 7C88-9BEB Directory of C:\Users\hsmyers\Perl_Dev\Chess PAUSE\PGN\Chess-PGN-EPD- +0.23\CSV 10/02/2010 09:44 AM <DIR> . 10/02/2010 09:44 AM <DIR> .. 04/02/2002 08:12 PM 196,290 ECO.txt 04/02/2002 07:39 PM 90,173 NIC.txt 04/02/2002 08:12 PM 467,540 Opening.txt 3 File(s) 754,003 bytes 2 Dir(s) 58,299,342,848 bytes free C:\>wc *.txt 2786 11138 196290 ECO.txt 1247 6231 90173 NIC.txt 4661 34709 467540 Opening.txt 8694 52078 754003 total

I'm not particularly hung up on difficulty of interface, but I do want reasonable response time for queries. I'm just beginning to think about the following:

  • DBD::SQLite
  • DBM::Deep
  • some sort of tied hash DB
  • ?

So the question becomes 'What are your suggestions?'

--hsm

"Never try to teach a pig to sing...it wastes your time and it annoys the pig."

Replies are listed 'Best First'.
Re: Least impact DB for CPAN modules
by BrowserUk (Pope) on Oct 02, 2010 at 20:22 UTC

    Given the modest size of those files, and the fact that your application (apparently) only needs hash-style, key-value access, I wonder why you're using a DB at all.

    The simplest and fastest--both loading and access--mechanism would be Storable::nstore() & retrieve()

    It's simple, fast, cross-platform and core. It imposes minimum prereqs upon your users and requires no "glue" code for your application.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

      Old habits concerning memory usage I suppose. Not really sure why I got on the DB train when I began. Given the 22 previous versions, it was a while ago! I'd just looked at Storable for another application and I must confess it didn't even register as a possibility for this problem— but from what I remember, it would probably have the least impact on the existing code and would also result in a speed up...oh hurt me again!!

      --hsm

      "Never try to teach a pig to sing...it wastes your time and it annoys the pig."

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://863099]
Approved by ww
help
Chatterbox?
[stevieb]: nice! I just finished a GPS "take me home" device last week, and did a bunch of software updates to it yesterday. I also created a pseudo chip with an Arduino to simulate an IC, where it responds to register read/writes over the I2C bus...
[stevieb]: ...from an I2C master. It's ugly and there are many changes I'm going to make, but I had not done anything like it before. It's designed for my RPi:: automated test platform; a system that does CI on *all* my RPi modules.
[shmem]: pseudo chip?
[stevieb]: well, what happens is the Arduino 'listens' for requests r/w, and does the appropriate thing when it's interrupted based on the 'register' address sent in. It's ugly as it was my first attempt, but I've got great new ideas I'm just sitting.
[stevieb]: ...down to implement now. Here's the sketch as it currently sits
[shmem]: well I use I2C and SPI and stuff, but creating a pseudo chip looks to me like lot of indirection and memory clutter... not?
[choroba]: Are you going to use the device soon? Related to your comment about "not having much time to do a lot of coding"...
[stevieb]: sure, but I'm just learning ;) I consider it practice to get a good understanding of what goes on *after* an I2C/SPI request is made
[shmem]: ah ok. Gonna read that. but now....
shmem compiles himself into his template

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (9)
As of 2017-06-25 22:42 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    How many monitors do you use while coding?















    Results (572 votes). Check out past polls.