Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Re: Loading large amount of data in a hash

by maverick (Curate)
on May 01, 2002 at 22:22 UTC ( #163413=note: print w/replies, xml ) Need Help??


in reply to Loading large amount of data in a hash

Right idea. You just have a few little details. You can't natively store references in dbms. You'll have to serialize the arrayrefs down to scalars before you can store them, using something like Storable. Which leads to the second issue. If memory serves, GDBM has a fixed size for those scalars...Berkely BTrees do not. Try something like:
use DB_File; use Storable; tie %data_parsed, "DB_File", "$hashfil", O_RDWR|O_CREAT, 0666; # inside read data loop $data_parsed{$key} = freeze($array_ref); # inside use data loop $array_ref = thaw($data_parsed{$key});
DB_File is slow to create, but fast to read. That may or may not be an issue.

HTH

Update

Ya know...after taking another glance at this, 800 is a lot of data. Plus, you have a key, and array combination. Perhaps it's time to move up to a full blown database like MySQL or Postgres?

/\/\averick
OmG! They killed tilly! You *bleep*!!

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://163413]
help
Chatterbox?
[ovedpo15]: another question about regex. if I have a string like this: "a/b/,c/d/e,5" I would like to remove the string after the last comma so the output will be "a/b/,c/d/e" how to do so?
[Corion]: ovedpo15: You could split on comma, and then join everything together, except the last part. Or you could match and make sure that the last part has no comma, like qr!,[^,]+$!, or you could use rindex.
[ovedpo15]: But I don't know the length of the string. will it still work?
[ovedpo15]: The only thing I know is that there is a comma at the end and a value after that
[hippo]: Try it and see. Your computer won't explode.
[Corion]: length will tell you the length of a string.
[Corion]: But if you are parsing CSV, again, let me point to Text::CSV_XS, which you should use instead
[hippo]: +1 for Text::CSV_XS
[ovedpo15]: Im not allowed to use additional modules, so I can't use it. looking for a simple regex to do this task though..

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (9)
As of 2018-05-27 10:36 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Notices?