Re: Data structures benchmark(pack vs. arrays vs. hashes vs. strings)

[BrowserUk]

First, I get a different results for the construction with pack coming out fastest. These are the results of your benchmark (unchanged except for shortening the names) on my system:

c:\test>942747.pl
           Rate   hash  comma  array packed
hash   186995/s     --   -27%   -76%   -77%
comma  255572/s    37%     --   -68%   -69%
array  789101/s   322%   209%     --    -4%
packed 819141/s   338%   221%     4%     --
            Rate  comma packed   hash  array
comma   273531/s     --   -62%   -70%   -86%
packed  714529/s   161%     --   -21%   -63%
hash    900220/s   229%    26%     --   -53%
array  1925183/s   604%   169%   114%     --

Now to your questions:

1. Why is packing and returning six values slower than constructing a whole Perl AV data structure, creating each SvPV for each position, copying all the elements inside it and then returning that ? (compare the times of construct_packed with construct_array)

   Your construct_array doesn't "Perl AV data structure, creating each SvPV for each position, copying all the elements inside it".

   It simply returns a reference to the existing array that you constructed to hold the values.

   But you also construct that same array inside each of the other construct_* benchmark subs before getting to the additional bit they have to do.

   So the surprise -- on my system -- is that the pack version manages to be quicker. Even though it is only a little, it is consistent.

   Ie. construct_packed() does everything that construct_array() does, and then a little bit more, and still comes out quicker.

   This is a sure sign that you're not benchmarking what you think you are.

2. Why is unpacking a blob of 6 values to 6 different SV structures(among which 2 SvIV and 3 SvPVNV and one SvUV ) slower than dereferencing a whole array and copying contents to 6 different scalars(the same types mentioned before) ? (compare times of access_array and access_packed)

   Again, both access_array() and access_packed() are allocating six scalars to hold the results and copying the values retrieved into them.

   But access_packed() has also to first to: a) parse the template string; b) allocate scalars (on the stack) to hold the unpacked values; c) unpack those values. How could it be quicker?

3. Why is accessing a hash faster than just unpacking a string filled with data and accessing the contents ? (compare access_packed and access_hash)

   Again, access_packed() also has to allocate the six (actually seven, but you asked us to ignore that), scalars to hold the results. But whereas access_hash() only has to dereference a hash ref and copy the scalar; access_packed() has to: a) call a subroutine (unpack); b) parse the template; c) allocate 7 more scalars on the stack; d) assign the values extracted from the packed scalar to those stack vars; e) copy the values to the named scalars in the caller.

The bottom line is that both your benchmark and your expectations are fundamentally flawed.

Your expectations because you are imagining that pack & unpack should be doing less work, but in fact they are doing more. Accessing native arrays and hashes are "native operations", they don't require calling to a subroutine; setting up and tearing down teh stack frames for that subroutine; or parsing the templates as required by pack & unpack.

Your benchmark is flawed because you are attempting to benchmark very small operations, but ignoring the fact that all the setup required to benchmark those operations -- initialising the input arrays, calling the benchmark subroutines, constructing the scalars to hold the return values etc. -- is a) common to all the benchmarks; b) swamping the time taken to perform the bit you are trying to test.

I sympathise with the need to optimise your code, but you are going about it in the wrong way. If you'd care to post the actual code; or perhaps a cut down but working example that demonstrates the part that is taking too long, then you'd likely get some very good help in optimising it. In the not so distant past we've seen one or two orders of magnitude improvements in code posted here, that comes just from doing things slightly differently. (As opposed to major algorithmic improvements which can yield even greater savings.)

[spx2]

I am very surprised you got those resuls. I haven't tried the code on Windows, but yes, it's a bit bizarre that only the second report looks the same as on my machine. The targeted operating system for this code is Linux so I think I will try it on multiple Linux machines to see wether there's any difference. But Windows should've had the same result really.. :|

1. yes that's true, but please take into consideration that if I'm doing that every time, then when comparing the times of the benchmarks, each of them has this operation inside them, so the order is preserved because I added the operation in each of the scenarios. does this sound like a sane assumption ? I hope it does, but I'm not an expert so maybe I'm wrong

2.

• a) I fully agree, this might be the main cause. However, in the case of assigning a value to a scalar, isn't there some "typechecking" done to see if it's gonna be stored in a SvIV, SvUV,SvPV,SvPVNV. I'm talking about the fact that Perl would need to guess which type exactly the data is, in order to know what kind of data structure it will allocate for it. Whereas if you use unpack, you already tell it what kind of data it's gonna get, so shouldn't that be faster ?
• b) this is done in both access_array and access_packed so it should take about the same amount of time in both
• c)this would just need to do like a memcpy with a given number of bytes from one place to the other, keeping a pointer on the data and just galloping over the data right ? so it shouldn't be so time-consuming

3. c),d),e) so these are not the ones on the LHS of unpack, these are additional ones for the stack ! hmm, any way to avoid that ?

[sundialsvc4]

An excellent and very thorough analysis . . . up-voted.