Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re: Minimum Hash keys allocated?

by cdarke (Prior)
on Sep 27, 2010 at 10:10 UTC ( #862174=note: print w/ replies, xml ) Need Help??


in reply to Minimum Hash keys allocated?

You need a huge number of keys to make pre-allocation worth it. Running this several times:

use warnings; use strict; use Benchmark; my $start = new Benchmark; my $size = 10000000; my %hash; keys(%hash) = $size; for my $i (1..$size) { my $key = "key$i"; $hash{$key} = $key; } my $end = new Benchmark; my $diff = timediff ($end, $start); print "The code ran for \n",timestr($diff),"\n";
With 10,000,000 keys, typical numbers for pre-allocated hash keys was:
19 wallclock secs (17.36 usr + 0.94 sys = 18.30 CPU)
Typical numbers with the pre-allocation commented out:
21 wallclock secs (21.23 usr + 0.63 sys = 21.86 CPU)

I had to use 100,000 keys to even measure a difference, which was typically 0.03 seconds.

It ain't worth it.


Comment on Re: Minimum Hash keys allocated?
Select or Download Code
Re^2: Minimum Hash keys allocated?
by flexvault (Parson) on Sep 27, 2010 at 11:01 UTC

    I can't show a simple script to defend my earlier claim, since the times were taken using Time::HiRes qw(gettimeofday).

    The sub-system used is 20,000+ lines of code running persistently with pre-forked servers that dynamically expand/contract on activity. The servers respond to tcp calls from a web server with a very small mod-perl cgi script preloaded.

    The design goal was to support 3 transactions per second per core with enough memory to prevent paging.

    One bottleneck was to reload all hashes required for each transaction. Testing found that pre-allocting the hash key number had a significant improvement in production.

    The last stress test, we exceeded 3.52 tranactions per second per core. However, the difference between hardware, operating systems, web-servers, mod-perl, and versions of perl is significant, so your mileage may vary!

    Thank you

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://862174]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (7)
As of 2014-11-26 22:02 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My preferred Perl binaries come from:














    Results (174 votes), past polls