Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Re: Disk based hash (as opposed to RAM based)

by techtruth (Novice)
on Oct 07, 2012 at 22:57 UTC ( #997724=note: print w/ replies, xml ) Need Help??


in reply to Disk based hash (as opposed to RAM based)

I have looked around more and found http://search.cpan.org/~chorny/MLDBM-2.04/lib/MLDBM.pm

This seemed to do the trick with a bit of modification to my code. However now an error 22 is output, which means that the values I am storing under keys is too large. I suppose it is worth mentioning that I my data is "wide" in the sense that each key will have 1000 or so values that I hoped to store on the array.

use MLDBM; # this gets SDBM and Data::Dumper use Fcntl; # to get them constants my $dbm; my %hash; unlink ('testmldb.dir'); unlink ('testmldb.pag'); $dbm = tie %hash, 'MLDBM', 'testmldbm', O_CREAT|O_RDWR, 0640 or di +e $!; while($html =~ m/example(key)regex(value)example/ig ) { my $tempArray = $hash{$key}; push( @{ $tempArray } , $value); $hash{$domain} = $tempArray; undef $tempArray; #Just for fun-zies }

MLDBM docs do say something about berkleyDB not having this limit, because of how it stores the values. (MLDBM apparently uses joined strings as my array instead of a real array) I will look into the berklyDB approach... I was hoping to avoid using a db and just a file, because I might as well just dump it into a temporary table in my database (which the program also uses) I would just like to take advantage of perls amazing hashes if I could.


Comment on Re: Disk based hash (as opposed to RAM based)
Download Code
Re^2: Disk based hash (as opposed to RAM based)
by davido (Archbishop) on Oct 08, 2012 at 00:56 UTC

    I think BrowserUk's mention of DBM::Deep is on track. It should be able to handle whatever level of depth you want to throw at it, and arbitrarily long key lengths. The fact that it's written in pure-Perl is a good feature, as its compatibility with Perl's flexible data types should be higher than is sometimes symptomatic of solutions built in the more rigid environment of lower-level statically typed languages.


    Dave

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://997724]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (11)
As of 2014-09-18 12:39 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (114 votes), past polls