Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw

Uniq and sort in perl

by slayedbylucifer (Scribe)
on Jul 12, 2012 at 08:43 UTC ( #981329=perlquestion: print w/replies, xml ) Need Help??
slayedbylucifer has asked for the wisdom of the Perl Monks concerning the following question:

I am fetching two columns from oracle DB using the DBD::Oracle module. this is my code whihc Actually fetches the output:

my $sth = $dbh->prepare($query); $sth->execute(); while ( my $row_ref = $sth->fetchrow_arrayref ) { my $vm = $row_ref->[0]; my @nfs = split (/:/, $row_ref->[2]); my $filer = $nfs[0]; print "$vm, $filer, \n"; }

I have omitted rest of the code but above while loop should get you the idea what I am trying to do. "$query" value was given to me by the Oracle DBA and it is fairly complicated as it does quite a few comparisons. BTW, i dont udnertand how that query is wirtten. Anyway, it fetcehs the information that I need.

The problem is, the output of above "while" loop gives me few duplicate entries as below:,,,,,,,,,,,,,,,,

Pelase note that the Query statement has the "unique" function running. But after I run the "split" command in while loop, I get above few duplicate entries.

So, How to sort these entries and make them unique. I mean above, above duplicate entries should look like below:,,,,,,

please guide me on a technique which can be used in above "while" Loop. In bash, it is fairly simple to run ...| sort | uniq . but I could not find a way in perl to do the same


Replies are listed 'Best First'.
Re: Uniq and sort in perl
by Utilitarian (Vicar) on Jul 12, 2012 at 08:59 UTC
    Any time you need to get a unique list in Perl you should think of using a hash. Say you defined a hash called %seen outside the while loop and changed the print line to read...
    print "$vm, $filer, \n" if ! $seen{"$vm:$filler"}++;
    This will print the entry the first time and increment the seen flag for that pair, now subsequent identical results will encounter a set flag and not print as a result

    print "Good ",qw(night morning afternoon evening)[(localtime)[2]/6]," fellow monks."

      Utilitarian, I can't thank you enough. Your solution is **exactly** what I was looking for. Thank you very Much !!

      But, unfortunately i am not able to decode or understand your oneliner. Could you guide me to some documentation for the same.

      Thanks Again for the lightning fast response.

        What is happening in that one liner:
        print "$vm, $filer, \n" if ! $seen{"$vm:$filler"}++

        It's quite an elegant piece of code, with a lot going on. So let's break it down:

        • The hash called '%seen' is being checked - to see if it contains a key called "$vm:$filler". The value of this is being tested for being 'false'. (because there's 'if !' which is 'if not') - which is literally 'if not seen'
        • So if $seen{"$vm:filler"} is _not_ 'true' - we haven't seen it, and so we print it.
        • And then that ++ kicks in, to post increment, 'adding one' to $seen{"$vm:$filler"}' - so the first time we try doing that, it's going to be undefined (and thus false) triggering a print. Second time, it'll be 'non zero' and so evaluate as 'true'.
        For bonus points, you can probably do:
        foreach my $key ( keys %seen ) { print "$key was seen $seen{$key} times\n"; }
        Does that make it any clearer?
Re: Uniq and sort in perl
by johngg (Abbot) on Jul 12, 2012 at 09:40 UTC

    Utilitarian has given you a method of printing unique entries but if you also want to sort them then further steps will be required. Instead of printing in your while loop, declare an array outside of the loop and push your lines onto it inside the loop. You can then print your sorted unique lines afterwards. I use the inet_aton() from Socket to translate the IP numbers to a sortable string and use a Schwartzian Transform to do the sorting.

    use strict; use warnings; use Socket; my @lines = split m{(?<=\n)}, <<EOD;,,,,,,,,,,,,,,,, EOD print do { my %seen; map { $_->[ 0 ] } sort { $a->[ 2 ] cmp $b->[ 2 ] || $a->[ 1 ] cmp $b->[ 1 ] } map { my @flds = split m{,\s*}; [ $_, $flds[ 0 ], inet_aton( $flds[ 1 ] ) ] } grep { not $seen{ $_ } ++ } @lines };

    The output.,,,,,,,,

    I hope this is helpful.



      Johngg, thanks for the response. your solution is way more detailed and I will definitely refer it in complex scripts where I would need sort and uniq...Thanks.
Re: Uniq and sort in perl
by Anonymous Monk on Jul 12, 2012 at 08:55 UTC
Re: Uniq and sort in perl
by thargas (Deacon) on Jul 12, 2012 at 14:41 UTC
    You might want to consider whether the SQL is correct or not. If it's supposed to return unique info, the fact that it doesn't sounds suspicious to me. While the various other answers may allow you to see only unique info, this may be covering up a more basic problem.
      This is a very valid point, selecting only the fields you are interested in would improve performance as well as being the "correct" way to achieve this

      print "Good ",qw(night morning afternoon evening)[(localtime)[2]/6]," fellow monks."
        No. the SQL query is not giving me the duplicate entries. They look duplicate only after I run the "split" function inside my while loop. Thanks for your time.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://981329]
Front-paged by Corion
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (10)
As of 2018-06-21 09:28 GMT
Find Nodes?
    Voting Booth?
    Should cpanminus be part of the standard Perl release?

    Results (117 votes). Check out past polls.