Checking if an item exists in an array before adding a record to it.

[viffer]

Hi people..

I'm populating an array, which I then use in a foreach statement to print into an html table.

The issue I have is that I'm getting duplicates in the array, so basically I want to check if the item exists in the array and then only add the new record if it doesn't exist.

I've tried turning the array into a hash, which works, but it seems to be a waste of time to drop the array into a hash every single time I need to check it.

        if ($server eq 'lsrch1') {
            my %params = map { $_ => 1 } @lsrch1;
            if(! exists($params{$string})) {
                push (@lsrch1, $string);
            }
        }

Is there a quicker, more efficient way to do it? I suspect using grep would be even slower.

Thanks all

[Old_Gray_Bear]

If you find that your choice of data-structure is getting in the way of writing clean logic, then change structure!

Use a hash from the outset, rather than gin up some convoluted check-for-existence routine that converts from arrays to hashes and back and forth. Once you have your data in a hash, you can print it easily enough in what ever order is most appropriate. (I am very fond of the sort(keys(%my_hash)) .... idiom in a foreach loop.)

(Written in the hour BC -- Before Coffee, please excuse the typoes).

[AnomalousMonk]

I'm populating an array, which I then use in a foreach statement to print into an html table.
The issue I have is that I'm getting duplicates in the array ...

If the array doesn't change during the process of populating the HTML table, why not unique-ify the array with List::MoreUtils::uniq before processing it? (Update: I.e., don't worry about adding dups.) (Original ordering is preserved by uniq.)

>perl -wMstrict -le
"use List::MoreUtils qw(uniq);
 ;;
 my @ra = qw(one spam spam spam two spam three four spam spam five);
 ;;
 my @unique = uniq @ra;
 print qq{@unique};
"
one spam two three four five

[RichardK]

Turning your array into a hash is going to visit each member of the array and have to do all the memory management for the hash so my guess is that it will be slower than grep. But benchmark it if you really want to know.

You could use first from List::Util so something like :-

   push @array,$value unless first {$_ eq $value} @array;

Or push all your values to the array and then get just the unique values using uniq from List::MoreUtils

[Happy-the-monk]

I suspect using grep would be even slower.

• either use grep on the array if you want to check if something's already in it as you suggest yourself
  
• or throw the dupes out when you are done populating the array (if order does not matter)
  
• or (again if order does not matter) use a hash all the way through and use keys %hash to get a list of unique entries instead of the array.

Cheers, Sören

[Laurent_R]

If the order in which you are populating the array is important, then populate both an array and an %already_seen hash (don't populate the array if the element exists in the hash). This will be much faster than greping your array each time.

If the order of insertion is not important, then don't use an array, use only a hash, you are guaranteed against duplicates. You can still sort your hash on keys.

Well, if the order is important, you might still use only a hash, with the elements as keys, and the order of insertion as values. At then end, you only need to sort the hash on the values. This will probably not be faster than populating both an array and a hash, but if speed is not crucial (no huge data set), the code will be a bit simpler.

[kcott]

G'day viffer,

If you declare %params earlier in your code, you can write:

        if ($server eq 'lsrch1') {
            my %params = map { $_ => 1 } @lsrch1;
            if(! exists($params{$string})) {
                push (@lsrch1, $string);
            }
        }

as:

        if ($server eq 'lsrch1') {
            push @lsrch1, $string unless $params{$string}++;
        }

You haven't shown the context of your code (which is fine); however, it leaves me in the dark as to advising where exactly to delare %params. Also, I don't know where $string comes from. Here's my test showing how my code might be implemented (adapt for your context as appropriate):

$ perl -Mstrict -Mwarnings -e '
    my @strings = qw{a a a b b c d d e e e};
    my $server = "lsrch1"; # Fudge for demo
    my @lsrch1;
    my %params;

    for my $string (@strings) {
        if ($server eq "lsrch1") {
            push @lsrch1, $string unless $params{$string}++;
        }
    }

    print "\@strings:     @strings\n";
    print "\%params keys: @{[sort keys %params]}\n";
    print "\@lsrch1:      @lsrch1\n";
'
@strings:     a a a b b c d d e e e
%params keys: a b c d e
@lsrch1:      a b c d e