Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

comment on

( [id://3333]=superdoc: print w/replies, xml ) Need Help??
you missed vr's point about hash slices.

Not so much "missed it" as chose not to address it.

With @h{ @keys } = @values; perl could use the size of the keys list to preallocate the hash

The problem with that is that in order to gain the 18% saving from the preallocation of my 10,000,000 key hash, the hash slice would:

  1. Allocate stack space to accommodate 20,000,000 items on the stack.
  2. Turn the 10,000,000 element array, @keys, into a 10,000,000 item list on the stack.
  3. Turn the 10,000,000 element array, @values, into a 10,000,000 item list also the stack.
  4. Process those two lists in parallel adding each key/value pair individually; doubling (and redoubling), hashing (and rehashing) as required.
  5. Free the space required to accommodate the 20,000,000 items on the stack back to the memory pool.

It's a lot more complicated than that, but it is sufficiently accurate to make my point.)

So, now the process not only has two copies of all the keys and values (in the arrays as well as the hash), it also has allocated enough space to store them all again on the stack.

So that means the peak memory usage required to do the slice assignment is at least 4 times more than is eventually required to construct the hash itself; and the hash was not preallocated either!

Slice assignment can be more efficient for relatively small hashes provided it is necessary to the rest of the code to have the keys and values stored in arrays as well. But if you are loading them into arrays, simply so that you can use list assignment to construct a hash from them, it is a disaster in terms of both memory usage and processing time.

And for anything larger then pretty modest sized hashes ( a few 10s or low 100s of key/value pairs) then the cost (memory and time) of building the two lists on the stack far outweights any gain from moving the loop into C. Especially as the potential of having the hash pre-sized doesn't happen. (It probably could, but it doesn't.)


With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority". The enemy of (IT) success is complexity.
In the absence of evidence, opinion is indistinguishable from prejudice.
.

In reply to Re^3: Does "preallocating hash improve performance"? Or "using a hash slice"? by BrowserUk
in thread Does "preallocating hash improve performance"? Or "using a hash slice"? by vr

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others contemplating the Monastery: (5)
As of 2024-04-26 08:33 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found