First off, as davorg puts it, you might consider caching to speed up the process. Then, since you seem to be sure you $a->{ip} is really an IP address, you might want to avoid the cost of firing the regexp engine.
I don't know if this would be a lot quicker, but you can easily turn an IP into a 32-bits integer using and eval and the v notation, which would allow you to compare two integers instead of four times two:
my $a32 = eval "v$a->{ip}";
my $b32 = eval "v$b->{ip}";
$a32 <=> $b32 ||
...
--bwana147
