Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Re: Minimum Hash keys allocated?

by cdarke (Prior)
on Sep 27, 2010 at 10:10 UTC ( #862174=note: print w/ replies, xml ) Need Help??


in reply to Minimum Hash keys allocated?

You need a huge number of keys to make pre-allocation worth it. Running this several times:

use warnings; use strict; use Benchmark; my $start = new Benchmark; my $size = 10000000; my %hash; keys(%hash) = $size; for my $i (1..$size) { my $key = "key$i"; $hash{$key} = $key; } my $end = new Benchmark; my $diff = timediff ($end, $start); print "The code ran for \n",timestr($diff),"\n";
With 10,000,000 keys, typical numbers for pre-allocated hash keys was:
19 wallclock secs (17.36 usr + 0.94 sys = 18.30 CPU)
Typical numbers with the pre-allocation commented out:
21 wallclock secs (21.23 usr + 0.63 sys = 21.86 CPU)

I had to use 100,000 keys to even measure a difference, which was typically 0.03 seconds.

It ain't worth it.


Comment on Re: Minimum Hash keys allocated?
Select or Download Code
Replies are listed 'Best First'.
Re^2: Minimum Hash keys allocated?
by flexvault (Prior) on Sep 27, 2010 at 11:01 UTC

    I can't show a simple script to defend my earlier claim, since the times were taken using Time::HiRes qw(gettimeofday).

    The sub-system used is 20,000+ lines of code running persistently with pre-forked servers that dynamically expand/contract on activity. The servers respond to tcp calls from a web server with a very small mod-perl cgi script preloaded.

    The design goal was to support 3 transactions per second per core with enough memory to prevent paging.

    One bottleneck was to reload all hashes required for each transaction. Testing found that pre-allocting the hash key number had a significant improvement in production.

    The last stress test, we exceeded 3.52 tranactions per second per core. However, the difference between hardware, operating systems, web-servers, mod-perl, and versions of perl is significant, so your mileage may vary!

    Thank you

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://862174]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (5)
As of 2015-07-30 04:19 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (269 votes), past polls