perlmeditation
AbigailII
I'm writing this because of "[id://349722Is this a fair shuffle?]".
In the thread,
[saintmike] asked whether a certain algorithm using splice was a fair
shuffle. [BrowserUk] replied that algorithm was fair (which it is  assuming
your random generator is fair), and that it was an implementation of
the FisherYates shuffle, which it isn't. FisherYates is an algorithm
that works in linear time  the presented algorithm takes quadratic
time (expected). For reference, the presented algorithm:
<blockquote><code>
my @a = (1..10);
my @b;
push @b, splice @a, rand @a, 1 while @a;
</code></blockquote>
The discussion resulted in the question why the presented algorithm is
quadratic, and [BrowserUK] writes:
<blockquote>
I'm still a little bemused by why swapping pointers on the linked list,
rather than swapping the contents of the elements the linked list points
at, becomes quadratic, but the (newer) numbers demonstrate your point. I
will have to sit down with a pen and paper and the source code of splice
to understand why the costs grow that way.
</blockquote>
[BrowserUK] focusses the wrong thing. The copying of the target element
isn't the issue. It's the splicing that's the bottleneck. There's no
"linked list" ([BrowserUK] is probably referring to the fact that the internal
array Perl uses stores pointers to SV's. Whether they are pointers, or
whether pointers are swapped or the content of the SV's isn't important
either  it's the splicing).
<p>
Consider the following array:
<pre>
++++++++++++
           
 0  1  2  . . .  k1  k  k+1  . . .  n2  n1  n 
           
++++++++++++
</pre>
To be able to do quick indexing, Perl stores the elements, just as done
is in C, in consecutive memory locations. It doesn't store the values
directly, just pointers to SVs, but that's not the point. Here's the
array again, but below are the memory locations of the elements (or
pointers to the elements), assuming each element takes 4 bytes.
<pre>
+ Array

v
++++++++++++
           
 0  1  2  . . .  k1  k  k+1  . . .  n2  n1  n 
           
++++++++++++
M M+4 M+8 M+4k4 M+4k M+4k+4 M+4n8 M+4n4 M+4n
</pre>
We've also introduced the <tt>Array</tt> pointer, which is the part of the
AV structure that points to the beginning of the array (element with index 0).
<p>
Now assume that we want to splice off element <tt>k</tt>. Our resulting
array will look like either of:
<pre>
+ Array

v
+++++++++++
          
 0  1  2  . . .  k1  k+1  . . .  n2  n1  n 
          
+++++++++++
M M+4 M+8 M+4k4 M+4k M+4k+4 M+4n8 M+4n4 M+4n
</pre>
or
<pre>
+ Array

v
+++++++++++
          
 0  1  2  . . .  k1  k+1  . . .  n2  n1  n 
          
+++++++++++
M M+4 M+8 M+4k4 M+4k M+4k+4 M+4n8 M+4n4 M+4n
</pre>
Now as you see, this requires that either <tt>k</tt> elements will be moved
(4 bytes 'up'), or that <tt>n  k  1</tt> elements will be moved (4 bytes
'down'). Now it's clear to see that in at least <tt>n/2</tt> cases, you have
to move at least <tt>n/4</tt> elements. In fact, if you do the math, you'll
see that the expected number of moves due to a single splice is about
<tt>n/4</tt>.
<p>
That's why splicing a single element out of an array takes linear time
(expected), and that's why the splicing shuffle algorithm takes quadratic time.
<p>
Abigail