Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

Comment on

( #3333=superdoc: print w/ replies, xml ) Need Help??
I'm writing this because of "Is this a fair shuffle?". In the thread, saintmike asked whether a certain algorithm using splice was a fair shuffle. BrowserUk replied that algorithm was fair (which it is - assuming your random generator is fair), and that it was an implementation of the Fisher-Yates shuffle, which it isn't. Fisher-Yates is an algorithm that works in linear time - the presented algorithm takes quadratic time (expected). For reference, the presented algorithm:
my @a = (1..10); my @b; push @b, splice @a, rand @a, 1 while @a;
The discussion resulted in the question why the presented algorithm is quadratic, and BrowserUK writes:
I'm still a little bemused by why swapping pointers on the linked list, rather than swapping the contents of the elements the linked list points at, becomes quadratic, but the (newer) numbers demonstrate your point. I will have to sit down with a pen and paper and the source code of splice to understand why the costs grow that way.
BrowserUK focusses the wrong thing. The copying of the target element isn't the issue. It's the splicing that's the bottleneck. There's no "linked list" (BrowserUK is probably referring to the fact that the internal array Perl uses stores pointers to SV's. Whether they are pointers, or whether pointers are swapped or the content of the SV's isn't important either - it's the splicing).

Consider the following array:

+------+------+------+-------+------+------+------+-------+------+------+------+
|      |      |      |       |      |      |      |       |      |      |      |
|   0  |   1  |   2  | . . . |  k-1 |   k  |  k+1 | . . . |  n-2 |  n-1 |   n  |
|      |      |      |       |      |      |      |       |      |      |      |
+------+------+------+-------+------+------+------+-------+------+------+------+
To be able to do quick indexing, Perl stores the elements, just as done is in C, in consecutive memory locations. It doesn't store the values directly, just pointers to SVs, but that's not the point. Here's the array again, but below are the memory locations of the elements (or pointers to the elements), assuming each element takes 4 bytes.
 
   +--- Array
   |
   v
+------+------+------+-------+------+------+------+-------+------+------+------+
|      |      |      |       |      |      |      |       |      |      |      |
|   0  |   1  |   2  | . . . |  k-1 |   k  |  k+1 | . . . |  n-2 |  n-1 |   n  |
|      |      |      |       |      |      |      |       |      |      |      |
+------+------+------+-------+------+------+------+-------+------+------+------+
 M      M+4    M+8            M+4k-4 M+4k   M+4k+4         M+4n-8 M+4n-4 M+4n
We've also introduced the Array pointer, which is the part of the AV structure that points to the beginning of the array (element with index 0).

Now assume that we want to splice off element k. Our resulting array will look like either of:

   +--- Array
   |
   v
+------+------+------+-------+------+------+-------+------+------+------+
|      |      |      |       |      |      |       |      |      |      |
|   0  |   1  |   2  | . . . |  k-1 |  k+1 | . . . |  n-2 |  n-1 |   n  |
|      |      |      |       |      |      |       |      |      |      |
+------+------+------+-------+------+------+-------+------+------+------+
 M      M+4    M+8            M+4k-4 M+4k   M+4k+4         M+4n-8 M+4n-4 M+4n   
or
           +--- Array
           |
           v
       +------+------+------+-------+------+------+-------+------+------+------+
       |      |      |      |       |      |      |       |      |      |      |
       |   0  |   1  |   2  | . . . |  k-1 |  k+1 | . . . |  n-2 |  n-1 |   n  |
       |      |      |      |       |      |      |       |      |      |      |
       +------+------+------+-------+------+------+-------+------+------+------+
 M      M+4    M+8            M+4k-4 M+4k   M+4k+4         M+4n-8 M+4n-4 M+4n   
Now as you see, this requires that either k elements will be moved (4 bytes 'up'), or that n - k - 1 elements will be moved (4 bytes 'down'). Now it's clear to see that in at least n/2 cases, you have to move at least n/4 elements. In fact, if you do the math, you'll see that the expected number of moves due to a single splice is about n/4.

That's why splicing a single element out of an array takes linear time (expected), and that's why the splicing shuffle algorithm takes quadratic time.

Abigail


In reply to Be aware of splice by Abigail-II

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • Outside of code tags, you may need to use entities for some characters:
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others surveying the Monastery: (4)
    As of 2014-12-25 17:52 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?

      Is guessing a good strategy for surviving in the IT business?





      Results (161 votes), past polls