Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Remove duplicate strings from an array

by Anonymous Monk
on Mar 13, 2007 at 14:40 UTC ( #604547=perlquestion: print w/ replies, xml ) Need Help??
Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hiya.

OK, maybe im looking in the wrong places or something, I found loads on removing duplicates from arrays but the examples they gave only seemed to work for integers.

One solution is to pop off each element of the array and compare it against every other element remaining. If there's a match, chuck it, otherwise it must be unique so add it to @output. Surely there's a more efficient perlesque way though?

So the idea is if I have an array:

@array = ("abc","def","abc","ghi","ghi","abc","jklm","abc","def")
The output should be:
@output = ("abc","def","ghi","jklm")
Can someone offer a Perl newbie a hand?!

Also can anyone suggest other good sites I can draw code samples/tutorials/information from?

Thanks!

Comment on Remove duplicate strings from an array
Select or Download Code
Re: Remove duplicate strings from an array
by Corion (Pope) on Mar 13, 2007 at 14:45 UTC

    This is a FAQ:

    perldoc -q duplicate

    How can I remove duplicate elements from a list or array?

    Your error likely stems from the fact that you're trying to use == as comparison operator, but == is the operator used to check for numeric comparison. If you want to compare two strings for equality as strings, use the eq operator. But it's better to use the method outlined in the FAQ.

Re: Remove duplicate strings from an array
by liverpole (Monsignor) on Mar 13, 2007 at 14:45 UTC
    A really quick way to do it is to create a hash, using the elements of the array as keys.

    The keys of a hash are always unique, so you just have to reconstruct the list of keys at the end:

    use strict; use warnings; my @array = ("abc","def","abc","ghi","ghi","abc","jklm","abc","def"); my %hash = map { $_ => 1 } @array; my @unique = keys %hash;

    s''(q.S:$/9=(T1';s;(..)(..);$..=substr+crypt($1,$2),2,3;eg;print$..$/
Re: Remove duplicate strings from an array
by davorg (Chancellor) on Mar 13, 2007 at 14:45 UTC
Re: Remove duplicate strings from an array
by agianni (Hermit) on Mar 13, 2007 at 14:47 UTC
    This sounds a lot like homework, so rather than giving you a solution, consider looking at the solutions that you said worked with integers and change your equality operators to those that compare strings.
Re: Remove duplicate strings from an array
by ferreira (Chaplain) on Mar 13, 2007 at 14:48 UTC

    This is almost a FAQ.

    sub uniq { my %h; return grep { !$h{$_}++ } @_ }
Re: Remove duplicate strings from an array
by johngg (Abbot) on Mar 13, 2007 at 15:14 UTC
    This is similar to liverpole's solution but populates the hash using a hash slice rather than a map. Also, you seem to want the resultant array sorted so I've done that.

    use strict; use warnings; use Data::Dumper; my @array = qw{abc def abc ghi ghi abc jklm abc def}; my %comb; @comb{@array} = (); my @uniq = sort keys %comb; print Data::Dumper->Dumpxs([\@uniq], [qw{*uniq}]);

    The output is

    @uniq = ( 'abc', 'def', 'ghi', 'jklm' );

    I hope this is of use.

    Cheers,

    JohnGG

Re: Remove duplicate strings from an array
by CountZero (Bishop) on Mar 13, 2007 at 15:39 UTC
    And using a module (List::MoreUtils):
    use List::MoreUtils qw(uniq); my @x = uniq 1, 1, 2, 2, 3, 5, 3, 4; # returns 1 2 3 5 4
    And this function preserves the order in the array (which the hash-based solutions don't).

    CountZero

    "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law

Re: Remove duplicate strings from an array
by Anonymous Monk on Mar 13, 2007 at 16:02 UTC
    I used perlfaq which suggested some of the code you guys have mentioned. Basically I get an output of the full list again...

    Here's my code, the file read in is a csv...

    my %defaultArgs = ( 'porder' => 1, 'ptitle1' => 2, 'ptitle2' => 3, 'ptitle3' => 4, 'yorder' => 5, 'xorder' => 6, 'mtitle' => 7, 'size' => 8, 'avail' => 9, 'select' => 10, 'symbol' => 11, 'invert' => 14, 'isvary' => 15, 'unit' => 16, 'default' => 17, 'min' => 18, 'max' => 19, 'relative' => 20, 'monitor' => 21, 'ac' => 22, 'en' => 23, 'summary' => 24, 'device' => 25 ); my @arguments = (); my @devices = (); my @unique = (); @devices = split( ',', $arguments[$defaultArgs{ device }] ); # foreach my $device (@devices) # { # print $device."\n"; # } my %hash = map { $_ => 1 } @devices; my @unique = keys %hash; foreach my $uni (@unique) { print $uni."\n"; }

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://604547]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (3)
As of 2014-09-22 02:34 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (177 votes), past polls