Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re^3: Store larg hashes more efficiently (10e6 md5s in 260MB at 4Ás per lookup)

by BrowserUk (Pope)
on Feb 27, 2013 at 01:00 UTC ( #1020788=note: print w/ replies, xml ) Need Help??


in reply to Re^2: Store larg hashes more efficiently (10e6 md5s in 260MB at 4Ás per lookup)
in thread Store large hashes more efficiently

Finally, it might be interesting to play with masking different amount of bits to see the space-saving vs. lookup time tradeoffs for different degrees of sparseness.

By all means; play.


With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.


Comment on Re^3: Store larg hashes more efficiently (10e6 md5s in 260MB at 4Ás per lookup)
Re^4: Store larg hashes more efficiently (10e6 md5s in 260MB at 4Ás per lookup)
by puterboy (Scribe) on Feb 27, 2013 at 03:28 UTC
    I would be happy to play... But before that I want to make sure that my two suggestions for improvement are valid since I am at best a Perl neophyte...

      I have my own projects to take care of. I've supplied you with code that measures both; and you have the problem.

      The only way for either of us to determine whether your suggestions are improvements; is to try them: do they make lookups slower or faster; or use more or less memory?

      If I do it, you remain a neophyte. If you do it, you become less of a neophyte.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
        I wasn't suggesting you try it for me - I just wanted to make sure I wasn't totally misunderstanding your code and go off on a wild goose chase...

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1020788]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (6)
As of 2014-09-19 06:34 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (132 votes), past polls