Re^6: RFC on Inline::C hack: Hash_Iterator

[demerphq]

My experience is largely with Other Languages that perhaps don't have such clever internals as Perl and where the underlying data structures can make the difference between terrible and acceptable performance speeds for extremely heavily loaded hash tables.

Im guessing that you mean scenarios where you have to hand code your own hash table implementations. Im also guessing that you mean scenarios where you have to work with statically sized hash tables. In such a scenario I could see your point for sure. A hash table of small threaded binary trees sounds like a good design to me.

However, just for your edification ill outline in general how perls hashes work: first, the size of the hash table is always a power of 2, starting at 8 elements large. When the bucket chain length starts getting too long (calculated I beleive by determining the ratio of the number of keys to the number of buckets) the size of the hash array is doubled and the keys of the original are remapped into the new hash array. The hash values are not recalculated as the power of two rule implies that the remapping can occur simply by anding a different bit mask with the hash values to determine the new slot in the array. In normal circumstances the actual keys are stored only once, in a master hash, with pointers from the buckets of the actual hash buckets to the master hashes buckets (which actually contain the key string). This key sharing is important as hashes are the most common way of representing objects which will by-and-large tend to have many keys in common.