How to remove duplicate key/value pairs in hash of array

[Tuna]

I thought that I was done, but no. Problem is, that I have ended up with a hash of arrays which contain duplicate values. I need to somehow process them, so that only unique key/value pair remains.

Example:

foreach $file(@list_of_files) {
       open FILE, "$dir/$$file" || die "Can't open $dir/$file: $!\n";
       while ($lines = <FILE>) {
           chomp $lines;
           next if ($lines =~ /unknown|shutdown/);
           ($name, $ranks, $serial_numbers) = split(' ',$line);
           push @{$HoL{$name}}, $ranks;     
 }
   }

My problem is that each file that I open contains some identical data. So, I wind up pushing duplicate key/value pairs into the hash of arrays. How can I avoid this, so I only end up with unique key/value pairs?

[chipmunk]

Unique key/value pairs? Sounds like you need a hash. ;)

foreach $file (@list_of_files) {
  open FILE, "$dir/$file" || die "Can't open $dir/$file: $!\n";
  while (defined($line = <FILE>)) {
    chomp $line;
    next if ($line =~ /unknown|shutdown/);
    ($name, $ranks, $serial_numbers) = split(' ',$line);
    $HoH{$name}{$ranks} = 1;     
  }
}

Think hash of hashes here, instead of hash of arrays.

[Tuna]

Here's how I solved my problem:

Beginning file:

 open (BEGIN_FILE, "$beg_file") || die  "Cannot open: $!\n"; 
   while ($line = <BEGIN_FILE>) {
     chomp $line;
     next if ($line =~ /unknown|shutdown|NONE/);
     ($as,$as_name,$nbr,$rtr,$int,$address,$int_name,$ifindex,$vpi,$vc
+i) = split(' ',$priv_line);
     $begin{$int_name} = $rtr;
   }

All the rest of the files:

foreach $file(@end) {
     open FILE, "$file" || die "Cannot open: $!\n";
     while ($line = <FILE>) {
       chomp $line;
      next if ($line =~ /unknown|shutdown|NONE/);
        ($as,$as_name,$nbr,$rtr,$int,$address,$int_name,$ifindex,$vpi,
+$vci) = split(' ',$priv_line);
     $end{$int_name} = $rtr;    
     }
   }

Then, in order to end up with a hash of arrays, containing files that exist in my "non-beginning" files, but not in the beginning file:

 my  %good_files = ();
   
   foreach  $key (sort keys %end) {
     if (!exists($begin{$key})){
       push (@{$good_files{$begin{$key}}}, ($key));
       
     }
   }

Works like a charm! Thanks to everyone for their help on this.

[chipmunk]

You've got an array of items that you don't want to contain duplicates. If you store those items as the keys of a hash instead of as the elements of an array, then you will never have duplicates. That's one of the features of a hash.

So, instead of a hash of arrays, my suggestion is that you use a hash of hashes. When you want to get the list of ranks for a specific name, you would do: my @ranks = keys %{$HoH{$name}};

[dws]

You can eliminate duplicate values by inverting the hash (turning key/value pairs into value/key pairs) and then re-inverting.

%hash = (a => 1, b => 1, c => 2);

%hash = reverse %hash; # key/value -> value/key
%hash = reverse %hash;

foreach $key ( keys %hash ) {
    print $key, " => ", $hash{$key}, "\n";
}

[bdalzell]

this works to remove duplicate values but if the keys are numeric it removes the lower numbered key value pairs. how would you remove the higher key numbered key value pairs retaining the key/value pair with the lowest value key?

[dfog]

Quickest way I can think of is to just compare the values in the array to check for dupes. A hash of hashes would most likely take less processor time, but require recoding quite a bit of the things you have already finished.

my $Flag = 0;
(/^$ranks$/)?$Flag++:"" foreach @{$HoL{$name}};
if ($Flag == 0) {
    push @{$HoL{$name}}, $ranks;
}

[hbm]

Tuna, A minor suggestion: You are bothering with far more items from split than you actually use. Instead of this:

($as,$as_name,$nbr,$rtr,$int,$address,$int_name,$ifindex,$vpi,
+$vci) = split(' ',$priv_line);

How about this:

($rtr,$int_name) = (split(' ',$priv_line))[3,6];