Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Re^2: sorting hash of array of hashes by value

by Special_K (Monk)
on Sep 04, 2014 at 21:49 UTC ( #1099597=note: print w/replies, xml ) Need Help??


in reply to Re: sorting hash of array of hashes by value
in thread sorting hash of array of hashes by value

That's pretty much what you've been doing, except you're also switching keys and values. I don't see a real reason for doing that, and it complicates your code (since values are not guaranteed to be unique, you have to maintain a list of keys for each), so I'd advise against that.



I was told that by switching the keys and values, it will improve performance during the sort operation because if I sort the values, I have to do two hash lookups for every comparison operation performed by the sort function. If I sort the keys, then I don't.

Therefore, if I really want to sort on the hash values, it would make sense to create a new temporary hash in which the values and keys are swapped, and then sort that new array on its keys (which are the values of the original hash), rather than the values. Is that incorrect?

For example, wouldn't this:



sort { $data_hash{$a} <=> $data_hash{$b} } keys(%data_hash)


be slower than this:

sort { $a <=> $b } keys(%reordered_data_hash)


Where %reordered_data_hash is %data_hash with the values swapped with the keys?

Replies are listed 'Best First'.
Re^3: sorting hash of array of hashes by value
by AppleFritter (Vicar) on Sep 04, 2014 at 22:24 UTC

    Therefore, if I really want to sort on the hash values, it would make sense to create a new temporary hash in which the values and keys are swapped, and then sort that new array on its keys (which are the values of the original hash), rather than the values. Is that incorrect?

    The answer, as usual, is "it depends". Of course extra hash lookups will slow things down, but by how much? Also (and this is where the "it depends" kicks in), in general you also have to factor in the time it'll take to construct a reverse hash.

    Here is a very simple test:

    #!/usr/bin/perl use strict; use warnings; use feature qw/say/; use Benchmark qw/cmpthese/; srand 0; our %hash = map { rand() } 1..1000; my $regular = sub { sort { $hash{$a} <=> $hash{$b} } keys %hash; }; my $keysonly = sub { sort { $a <=> $b } keys %hash; }; my $reverse_noref = sub { my %reverse_hash = (); foreach my $key (keys %hash) { $reverse_hash{$hash{$key}} = $key; } sort { $a <=> $b } keys %reverse_hash; }; my $reverse = sub { my %reverse_hash = (); foreach my $key (keys %hash) { push @{ $reverse_hash{$hash{$key}} }, $key; } sort { $a <=> $b } keys %reverse_hash; }; cmpthese(-2, { regular => $regular, keysonly => $keysonly, reverse => $reverse, reverse_noref => $reverse_noref });

    On the machine I'm currently on, this produces:

    $ perl 1099601.pl Rate reverse reverse_noref regular k +eysonly reverse 2222/s -- -35% -93% + -93% reverse_noref 3406/s 53% -- -89% + -89% regular 31693/s 1326% 830% -- + -0% keysonly 31726/s 1328% 831% 0% + -- $

    So in isolation, the difference between "regular" (with the hash lookup) and "keysonly" (without) is negligible (though of course the latter is ever so slightly faster), while constructing a reverse hash first is 13 times slower. Pushing to an array if/when you can't guarantee values are unique punishes you further, but even without that (reverse_noref) you're still an order of magnitude slower.

    What does that mean for you? If you construct a "reverse" hash as you go along, just like you'd construct a regular hash, there may not be much of a difference (or there may be; you'll have to check). If you already have a "regular" hash, just let sort do whatever it needs to do; the difference won't be as big if you only need to construct the reverse hash once and then access it many times, but it'll still be there.

    As always, it's better to measure than to assume when it comes to optimization. It may well be that your approach is actually faster for your script and data, and if speed is crucial, then using a less "natural" approach that's faster is entirely fair.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1099597]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (2)
As of 2021-10-17 01:17 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My first memorable Perl project was:







    Results (70 votes). Check out past polls.

    Notices?