Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation

Re^2: Problems with SDBM

by Laurent_R (Canon)
on Mar 15, 2013 at 14:37 UTC ( #1023712=note: print w/replies, xml ) Need Help??

in reply to Re: Problems with SDBM
in thread Problems with SDBM

Hi, thanks everyone for the answers already provided.

The main reason to tie is resource limits: the data input has about 30 million records (and slightly less than 2 GB) and that is just too large for a hash (untied hash, that is). Having said that, persistence would also be a bonus because later processes would use the same data and would not have to load it again. But persistence is not the primary reason for using tied hashes.

I am not too much concerned with speed performance at this point (although it might become important at some point, given the large data volume), my concern is that the process fails when I have loaded only about half of the data (15.8 million records), presumably because of the large volume of data. I could use several tied hashes to get around this volume limit, but that would be sort of awkward and unwieldy (and not very scalable).

It seems that the Berkeley DB is not available on our system, so it seems that it will not be an option.

Replies are listed 'Best First'.
Re^3: Problems with SDBM
by Anonymous Monk on Mar 15, 2013 at 15:11 UTC

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1023712]
[marto]: Raspberry Pi 3?
[Discipulus]: thanks under 50 neuros it's a good thing!
[Discipulus]: and it's GPIO how are seen by the OS? Perl can speak thougth them?

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (9)
As of 2017-10-17 09:51 GMT
Find Nodes?
    Voting Booth?
    My fridge is mostly full of:

    Results (225 votes). Check out past polls.