Re: Understanding the Schwartzian transform.

[tobyink]

The general form of the Schwartzian Transform is this:

my @sorted =
   map  { $_->[0] }
   sort { $a->[1] cmp $b->[1] } # or <=>, or $b before $a
   map  { [$_, f($_)] }
   @unsorted;

... where f($foo) is a comparatively expensive function.

For the purposes of this discussion, let's suppose that @unsorted is qw(8 11 9 4 3) and f($n) is a function that converts a number into Roman numerals. So we're sorting the numbers alphabetically by Roman numeral. We want the result qw(3 4 9 8 11) because 3 is "iii" and 11 is "xi".

Reading from bottom to top, the first transform map { [$_, f($_)] } converts each number to an arrayref where the first element is the original number, and the second is the Roman numeral for it:

# the first map does this
qw(8 11 9 4 3) --> (
  [ 8, "viii"],
  [11, "xi"],
  [ 9, "ix"],
  [ 4, "vi"],
  [ 3, "iii"],
)

Now we sort the transformed list alphabetically by $_->[1]:

# the sort does this:
(
  [ 8, "viii"],
  [11, "xi"],
  [ 9, "ix"],
  [ 4, "vi"],
  [ 3, "iii"],
) --> (
  [ 3, "iii"],
  [ 9, "ix"],
  [ 4, "vi"],
  [ 8, "viii"],
  [11, "xi"],
)

And the second transform map { $_->[0] }, just pulls out the original number from each arrayref:

(
  [ 3, "iii"],
  [ 9, "ix"],
  [ 4, "vi"],
  [ 8, "viii"],
  [11, "xi"],
) --> qw( 3 9 4 8 11 )

Let's see all that in action:

use strict;
use warnings;
use Roman ();

my @unsorted = qw(8 11 9 4 3);
*f = \&Roman::roman;

my @sorted =
   map  { $_->[0] }
   sort { $a->[1] cmp $b->[1] } # or <=>, or $b before $a
   map  { [$_, f($_)] }
   @unsorted;

print "@sorted\n";

__END__ # output shown below
3 4 9 8 11

The sharp-eyed may have spotted that there's a simpler way to do the same sort:

my @sorted = sort { f($a) cmp f($b) } @unsorted;

So why the transform? Well, it enables you to call f() fewer times; the function is called just once for each item on the list. Using the simpler version it's called twice for each pair of items that is compared, and there are typically more comparisons performed than items on the list. Let's have f() log how many times it was called. We'll define it like this:

*f = sub {
    $::count++;
    END { print "f() called $::count times\n" };
    goto \&Roman::roman;
};

Running this on my computer (and this may vary depending on the sorting algorithm that your version of Perl uses - it's changed once or twice), f() is called 10 times. With the transform it's called 5 times.

If we extend the original unsorted list to 100 items; e.g.:

use List::Util "shuffle";
my @unsorted = shuffle(1..100);

... then f() gets called typically over 1000 times, compared to exactly 100 for the version with the transform. If f() takes 0.01 seconds to run, that's a 10 seconds without the transform versus 1 second with the transform. If the unsorted list were 1000 items long, it would be about 3 minutes without the transform, versus 10 seconds with the transform.

(PS: Roman is available on CPAN; published by our fellow monk chorny.)