Unique Array Items

[Anonymous Monk]

I'm looking to take an array of numbers with loads of duplicates in it and return a sorted array of only one instance for each item in the original. That is, no matter how many times "133" shows up in the original, put it once and only once in the new array. I believe this should work:

@toBeMoved = sort {$a <=> $b} grep {$h{$_}++ == 1} @toBeMoved;

But it doesn't seem to. It's pulling through some items but certainly not all of them. Can anyone enlighten me as to why?

Thanks!

[tcf22]

Use ++$h{$_} instead of $h{$_}++ so the incrementing is done first, before the compare. $h{$_}++ returns the value, which is numerically 0 the first time through, then increments the value. This causes things seen only once to be ignored.

@toBeMoved = sort {$a <=> $b} grep {++$h{$_} == 1} @toBeMoved;

[Roy Johnson]

Or instead of "this is the first occurrence", use "haven't seen this before":

...grep {!$h{$_}++}...

[jasonk]

The '$h{$_}++ == 1' will never be true the first time $_ is seen, so anything that shows up only once in the array will be discarded by the grep.

[sauoq]

TIMTOWTDI and all that...

@toBeMoved = do { my %h; @h{ @toBeMoved } = (); sort keys %h };

[davido]

Ok, here's my solution. Instead of grep, it uses map to generate an anonymous hash, which is fed into keys, which is fed into sort....

@array = sort { $a <=> $b } keys %{ { map { $_ => 0 } @array } };

To me it's a pretty clear and readable solution, but others seem to prefer the grep alternatives. To each his own... that ought to be another Perl motto. ;)

---

Update: Here is a comparison of several of the methods layed out in this thread. My map method isn't the fastest, but I'll defend it on clarity...

Read more... (2 kB)

[graff]

You might try benchmarking the "do_map" case with:

... keys %{ map { $_ => undef }  @array };
                       # not 0

I actually don't know how it would compare in terms of speed, but it would use a fair bit less memory. (I just checked the process size of "=> 0" vs. "=> undef", and the latter was about 1 MB smaller on a set of 100K hash keys.)

[Art_XIV]

I'm suffering from idiom-dazzle! This works:


use strict;
my @toBeMoved = qw(133 22 103 133 133 22 1 999);
my %h;
$h{$_} = 1 for @toBeMoved;
@toBeMoved = sort {$a <=> $b} keys %h;
print "@toBeMoved\n";

But what's up with the increment and equality test inside of the grep? I'm reading this as each grep pass will set the value of the current hash element to 1 by the increment operator, but what's going on w/ the equality test? Would a kindly monk please break this grep block down for me?

[thelenm]

In this code:

my %h;
my @toBeMoved = sort {$a <=> $b} grep {++$h{$_} == 1} @toBeMoved;

the value associated with each element is incremented (not set to 1, that's important). So the grep block will only return a true value the first time a value is seen. E.g., the first time 133 is seen, ++$h{133} == 1 will be true. The second time 133 is seen, the block will return a false value, since $h{133} will now be 2, not 1. The block will only return a true value once for each unique value. Does that clear things up at all?

[Art_XIV]

That does clear it up! Thank you very much thelenm! :)

[matthewb]

It might be worth pointing out Array::Unique on CPAN which, if you want an array of unique values, transparently handles it for you.

Granted, you still have to sort the array but your problem has been reduced to a ten-second job with reusable components, which is often a good thing.

MB

[kelan]

How about this way:

my %items;
@items{@toBeMoved} = undef;
@toBeMoved = sort {$a <=> $b} keys %items;

Update: Guess I didn't read the other replies close enough. This is just a rewording of sauoq's reply above.