By doing
my ($b, $a) = ($a, $b)
you are not swaping, you simply defined another set of variables. Try this, it will clearly show you what happend, and why it is way fast:
$a = 100;
$b = 200;
my ($a, $b) = ($b, $a);
print $a, "\n"; #200
print $b, "\n"; #100
print $main::a, "\n";#100
print $main::b, "\n";#200
Update:
When you do:
($b, $a) = ($a, $b)
You are not working against two scalars, in stead you are working against two arrays, and two scalars. Not that it is slow, I don't even think it take less memory as you thought. Although you are not using any temps, Perl is using.