I suppose taking references might save memory, but it doesn't save any time over this (on small or large lists):
[snip]
I get about 16% better than the List::Util version (YMMV).

Indeed I was about to yell: "well, but then let's just adapt BrowserUk's version simply removing the referencing and dereferencing":

sub bzr (@) {
    my @a = @_;
    my $n;
    my $i = @_;
    map +($n=randi($i--), $a[$n], $a[$n]=$a[$i])[1], @_;
}
[download]

But that won't work any more! (So kids don't copy the code above, it's crap! ;-) OTOH if I compare your code (as 'run') with BrowserUk's, I get:

      Rate  run  buk
run 7467/s   -- -12%
buk 8454/s  13%   --
[download]

So that seems the best both speed and possibly memory wise.