Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Re^2: remove duplicates with hash and grep

by Smith (Initiate)
on Dec 22, 2014 at 22:04 UTC ( #1111094=note: print w/replies, xml ) Need Help??


in reply to Re: remove duplicates with hash and grep
in thread remove duplicates with hash and grep

Change it to this and am now getting the list but it has not removed the duplicates. Still lost

#!/usr/bin/perl use warnings; use strict; my $upload = "/var/tmp/work/upload"; my $work = "/var/tmp/work/STAP-domain_clean_project"; my $input3 = "$upload/domain.csv"; system ("dos2unix $input3"); open (IN,"$input3"); open (OUT,">>$work/local.rules"); while (<IN>) { chomp(); if ($_ =~ /^.+\.([A-Za-z0-9-_]+\.[A-Za-z]{2,})$/){ my @array = $1; my %seen = (); my @unique = grep { ! $seen{ @array }++ } @array; print "@unique\n"; } }

Replies are listed 'Best First'.
Re^3: remove duplicates with hash and grep
by GotToBTru (Prior) on Dec 22, 2014 at 22:18 UTC

    Your code redefines %seen and @array each time you read a line from your file. You get duplicates because your hash is always empty when you test it for a value.

    The following will produce a list of the unique values from IN:

    my (%seen,@unique); while (<IN>) { chomp; $seen{$1}++ if ($_ =~ /^.+\.([A-Za-z0-9-_]+\.[A-Za-z]{2,})$/); } @unique = keys %seen; printf "%s, ",$_ for @unique; print "\n";

    Updated for readability and coherence and typos (is it Monday already?)

    1 Peter 4:10

      Thank you! I understand now

Re^3: remove duplicates with hash and grep
by AnomalousMonk (Chancellor) on Dec 23, 2014 at 04:25 UTC
    ... now getting the list but it has not removed the duplicates.
    ...
        my @array = $1;

    On every iteration through the while-loop, this statement creates a new array and initializes it with a single element: the string that was captured to $1. This string is, of course, unique!

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1111094]
help
Chatterbox?
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (4)
As of 2018-06-24 00:02 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Should cpanminus be part of the standard Perl release?



    Results (126 votes). Check out past polls.

    Notices?