japhy has asked for the wisdom of the Perl Monks concerning the following question:
I've got a data munging problem that is confusing me. I have a list of domain names and their associated WHOIS data (company, contact, address, phone, fax, email). I want to group domains together if they share at least two of those values. So if abc.com's company and phone are the same as def.com's company and phone, they should be in a group (array/hash, not important). But I also want to group xyz.com with abc.com and def.com if xyz.com shares two values with abc.com or def.com, and they needn't be the same fields as before. So if xyz.com's email and address are the same as def.com's email and address, it should be lumped together with abc.com and def.com.
Right now, I've got a bunch of data structures, but I don't know how to allocate this data into the "islands" of related domains. The word island is used because, when this is all done, the groups of domains will not have two values in common with any other groups.
My data structures are:
- %domains whose keys are domain names. Each domain name's value in the hash is an array ref of (company, contact, address, phone, fax, email).
- %data whose keys are field names ('company', 'contact', 'address', 'phone', 'fax', 'email'). Each field's value in the hash is another hash reference...
- %{ $data{$field} } whose keys are distinct values for the specified $field. Each value in the hash is an array reference of domain names that held that particular value in that particular field.
UPDATE
I think I've found a very different approach. I'm going through the list of domain names and building a hash $link{$d1}{$d2} = $degree_of_linkage based on the number of fields domains $d1 and $d2 have in common. Then I can use that to build my islands.UPDATE
Here's the entirety of my code:#!/usr/bin/perl use strict; use warnings; chomp(my @fields = split /\t/, <>); my (%domains, %links, %rel, %group); shift @fields; while (<>) { chomp; my ($dom, @values) = split /\t/, $_, -1; @{ $domains{$dom} }{@fields} = @values; } my @dlist = my @domain_list = keys %domains; while (@dlist) { my $d1 = shift @dlist; for my $d2 ($d1, @dlist) { $links{$d1}{$d2} = [ grep { $domains{$d1}{$_} eq $domains{$d2}{$_} and $domains{$d1}{$_} ne "" and $domains{$d1}{$_} ne "Private, Registration" and $domains{$d1}{$_} ne "Domains by Proxy, Inc." and $domains{$d1}{$_} !~ /^DomainsByProxy.com/i } @fields ]; } } for my $d1 (@domain_list) { $rel{$d1} ||= $d1; $group{ $rel{$d1} }{$d1} = 1; for my $d2 (grep { @{ $links{$d1}{$_} } > 1 } keys %{ $links{$d1} }) + { $rel{$d2} ||= $d1; $group{ $rel{$d2} }{$d2} = 1; } } for my $d1 (sort { keys(%{ $group{$b} }) <=> keys(%{ $group{$a} }) } k +eys %group) { print "GROUPED TO $d1\n"; for my $d2 (sort keys %{ $group{$d1} }) { print " $d2 (via: @{ $links{$d1}{$d2} })\n"; } }
|
---|
Replies are listed 'Best First'. | |
---|---|
Re: Building "islands" of related data
by jfroebe (Parson) on Dec 08, 2005 at 17:46 UTC | |
Re: Building "islands" of related data
by Tanktalus (Canon) on Dec 08, 2005 at 17:50 UTC | |
Re: Building "islands" of related data
by BrowserUk (Patriarch) on Dec 09, 2005 at 01:02 UTC | |
by japhy (Canon) on Dec 09, 2005 at 16:18 UTC | |
Re: Building "islands" of related data
by TedPride (Priest) on Dec 09, 2005 at 00:19 UTC | |
Re: Building "islands" of related data
by Ryszard (Priest) on Dec 09, 2005 at 11:09 UTC |
Back to
Seekers of Perl Wisdom