Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask
 
PerlMonks  

Re: Memory Efficient Alternatives to Hash of Array

by Jenda (Abbot)
on Dec 28, 2008 at 20:42 UTC ( [id://732924]=note: print w/replies, xml ) Need Help??


in reply to Memory Efficient Alternatives to Hash of Array

Looks to me like you could pack the data quite a bit. The tags seem to contain just four letters (ACTG) ... which means you need just two bits per letter, that's 34*2=68 bits per tag which fits into 9 bytes instead of the original 34. Not sure what's allowed in the Error_rate_In_ASCII, but it looks there's quite a bit less than 256 possible characters in each position. So you could pack these as well.

This way you can save quite a lot of space and the comparison of the packed strings will also be quicker. Assuming the number of Error_rates for each Tag is not too big, it might also be better to use

$data{$packed_tag} .= $packed_rate . "\n";
instead of
push @{$data{$packed_tag}}, $packed_rate;
which will also let you use DB_File or some other on disk hash without the overhead of the multilevel ones like DBM::Deep.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://732924]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others cooling their heels in the Monastery: (6)
As of 2024-04-25 08:15 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found