Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Checking if an item exists in an array before adding a record to it.

by viffer (Beadle)
on Aug 10, 2013 at 14:59 UTC ( #1048909=perlquestion: print w/replies, xml ) Need Help??

viffer has asked for the wisdom of the Perl Monks concerning the following question:

Hi people..

I'm populating an array, which I then use in a foreach statement to print into an html table.

The issue I have is that I'm getting duplicates in the array, so basically I want to check if the item exists in the array and then only add the new record if it doesn't exist.

I've tried turning the array into a hash, which works, but it seems to be a waste of time to drop the array into a hash every single time I need to check it.

if ($server eq 'lsrch1') { my %params = map { $_ => 1 } @lsrch1; if(! exists($params{$string})) { push (@lsrch1, $string); } }
Is there a quicker, more efficient way to do it? I suspect using grep would be even slower.

Thanks all

  • Comment on Checking if an item exists in an array before adding a record to it.
  • Download Code

Replies are listed 'Best First'.
Re: Checking if an item exists in an array before adding a record to it.
by Old_Gray_Bear (Bishop) on Aug 10, 2013 at 15:32 UTC
    If you find that your choice of data-structure is getting in the way of writing clean logic, then change structure!

    Use a hash from the outset, rather than gin up some convoluted check-for-existence routine that converts from arrays to hashes and back and forth. Once you have your data in a hash, you can print it easily enough in what ever order is most appropriate. (I am very fond of the sort(keys(%my_hash)) .... idiom in a foreach loop.)

    (Written in the hour BC -- Before Coffee, please excuse the typoes).

    ----
    I Go Back to Sleep, Now.

    OGB

Re: Checking if an item exists in an array before adding a record to it.
by AnomalousMonk (Bishop) on Aug 10, 2013 at 18:26 UTC
    I'm populating an array, which I then use in a foreach statement to print into an html table.
    The issue I have is that I'm getting duplicates in the array ...

    If the array doesn't change during the process of populating the HTML table, why not unique-ify the array with List::MoreUtils::uniq before processing it? (Update: I.e., don't worry about adding dups.) (Original ordering is preserved by uniq.)

    >perl -wMstrict -le "use List::MoreUtils qw(uniq); ;; my @ra = qw(one spam spam spam two spam three four spam spam five); ;; my @unique = uniq @ra; print qq{@unique}; " one spam two three four five
Re: Checking if an item exists in an array before adding a record to it.
by RichardK (Parson) on Aug 10, 2013 at 15:17 UTC

    Turning your array into a hash is going to visit each member of the array and have to do all the memory management for the hash so my guess is that it will be slower than grep. But benchmark it if you really want to know.

    You could use first from List::Util so something like :-

    push @array,$value unless first {$_ eq $value} @array;

    Or push all your values to the array and then get just the unique values using uniq from List::MoreUtils

Re: Checking if an item exists in an array before adding a record to it.
by Happy-the-monk (Canon) on Aug 10, 2013 at 15:19 UTC

    I suspect using grep would be even slower.

    • either use grep on the array if you want to check if something's already in it as you suggest yourself
    • or throw the dupes out when you are done populating the array (if order does not matter)
    • or (again if order does not matter) use a hash all the way through and use keys %hash to get a list of unique entries instead of the array.

    Cheers, Sören

    Créateur des bugs mobiles - let loose once, run everywhere.
    (hooked on the Perl Programming language)

Re: Checking if an item exists in an array before adding a record to it.
by Laurent_R (Canon) on Aug 10, 2013 at 19:15 UTC

    If the order in which you are populating the array is important, then populate both an array and an %already_seen hash (don't populate the array if the element exists in the hash). This will be much faster than greping your array each time.

    If the order of insertion is not important, then don't use an array, use only a hash, you are guaranteed against duplicates. You can still sort your hash on keys.

    Well, if the order is important, you might still use only a hash, with the elements as keys, and the order of insertion as values. At then end, you only need to sort the hash on the values. This will probably not be faster than populating both an array and a hash, but if speed is not crucial (no huge data set), the code will be a bit simpler.

Re: Checking if an item exists in an array before adding a record to it.
by kcott (Bishop) on Aug 11, 2013 at 06:09 UTC

    G'day viffer,

    If you declare %params earlier in your code, you can write:

    if ($server eq 'lsrch1') { my %params = map { $_ => 1 } @lsrch1; if(! exists($params{$string})) { push (@lsrch1, $string); } }

    as:

    if ($server eq 'lsrch1') { push @lsrch1, $string unless $params{$string}++; }

    You haven't shown the context of your code (which is fine); however, it leaves me in the dark as to advising where exactly to delare %params. Also, I don't know where $string comes from. Here's my test showing how my code might be implemented (adapt for your context as appropriate):

    $ perl -Mstrict -Mwarnings -e ' my @strings = qw{a a a b b c d d e e e}; my $server = "lsrch1"; # Fudge for demo my @lsrch1; my %params; for my $string (@strings) { if ($server eq "lsrch1") { push @lsrch1, $string unless $params{$string}++; } } print "\@strings: @strings\n"; print "\%params keys: @{[sort keys %params]}\n"; print "\@lsrch1: @lsrch1\n"; ' @strings: a a a b b c d d e e e %params keys: a b c d e @lsrch1: a b c d e

    -- Ken

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1048909]
Approved by Perlbotics
Front-paged by toolic
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (5)
As of 2022-05-23 12:35 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Do you prefer to work remotely?



    Results (82 votes). Check out past polls.

    Notices?