Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

How to compare 3 lists

by shammow (Novice)
on Oct 25, 2012 at 10:16 UTC ( [id://1000803]=perlquestion: print w/replies, xml ) Need Help??

shammow has asked for the wisdom of the Perl Monks concerning the following question:

Hi, I have three servers which have a ZFS dataset replicated between them (one master two slaves) I need to compare the data collected (via ssh) on the three systems and clean up snapshots as long as there is overlap. The command returns a list of snapshots like

snapshot1 date-of-creation snapshot2 date-of-creation etc

What I want to do is compare the lists and if snapshots 1-6 exist on all three do something with the first four on the master. Getting the info and then doing something with the results is no problem, I'd just like some pointers on where to start comparing the outputs....

Thanks

Replies are listed 'Best First'.
Re: How to compare 3 lists
by roboticus (Chancellor) on Oct 25, 2012 at 11:00 UTC

    shammow:

    For comparing multiple lists, I frequently use a hash of bit vectors, where bit 0 would be for the first list, bit 1 for the second, etc. Something like this:

    $ cat lists.pl #!/usr/bin/perl use strict; use warnings; use 5.10.0; # Build a few lists my @lists; for my $i (0 .. 1+int(3*rand)) { # several lists to compare push @{$lists[$i]}, int 16 * rand for 1 .. 15; print "LIST $i: ", join(", ", sort {$a<=>$b} @{$lists[$i]}), "\n"; } # Build a scoreboard of which items are in each list my %tally; my $bit=0; for my $list (@lists) { for my $item (@$list) { $tally{$item} |= 1<<$bit; } ++$bit; } # Show which items were in all lists my $all = (1<<$bit)-1; print "There were $bit lists. Items found:\n"; for my $item (sort {$a<=>$b} keys %tally) { print "Item $item: "; if ($tally{$item} == $all) { print "in *all* lists\n"; next; } my @present; for ($bit=0; $bit<@lists; $bit++) { push @present, $bit if $tally{$item} & 1<<$bit; } print "in lists: ", join(", ", @present), "\n"; } $ perl lists.pl LIST 0: 1, 3, 4, 6, 8, 8, 9, 9, 10, 10, 11, 11, 11, 13, 14 LIST 1: 0, 0, 0, 0, 0, 3, 3, 3, 5, 6, 8, 11, 11, 15, 15 LIST 2: 0, 1, 3, 4, 6, 6, 6, 8, 8, 9, 9, 9, 12, 13, 15 LIST 3: 1, 4, 4, 5, 5, 6, 8, 8, 8, 8, 9, 9, 12, 12, 13 There were 4 lists. Items found: Item 0: in lists: 1, 2 Item 1: in lists: 0, 2, 3 Item 3: in lists: 0, 1, 2 Item 4: in lists: 0, 2, 3 Item 5: in lists: 1, 3 Item 6: in *all* lists Item 8: in *all* lists Item 9: in lists: 0, 2, 3 Item 10: in lists: 0 Item 11: in lists: 0, 1 Item 12: in lists: 2, 3 Item 13: in lists: 0, 2, 3 Item 14: in lists: 0 Item 15: in lists: 1, 2

    It's a pretty handy trick for comparing multiple lists. Of course, it doesn't keep track of how many times items are found in each list. Normally, though, that's not an issue. If you want to do something with *only* the items in all lists *and* you want to keep some data about the items, such as the number of times they appear in a list, or an associated data structure, you can do it more like:

    .... oops! my son is ready for school, this is unfinished code, so I'll fix it when I get to work:

    Update: OK, here's the finished version:

    $ cat lists_2.pl #!/usr/bin/perl use strict; use warnings; use 5.10.0; # Build a few lists my @lists; for my $i (0 .. 1+int(3*rand)) { # several lists to compare push @{$lists[$i]}, int 16 * rand for 1 .. 15; print "LIST $i: ", join(", ", @{$lists[$i]}), "\n"; } # Track the items found in each list, including the order in # which they appear my %tally; for my $listnum (0 .. $#lists) { # Get a list my $list=$lists[$listnum]; for my $itemnum (0 .. $#$list) { # Get next item from the list my $item = $$list[$itemnum]; # Each item slot (value in list) gets an array, # where each slot is associated with a list. # In that array is an array of indexes into the # list where each item is found push @{$tally{$item}[$listnum]}, $itemnum; } } # Show which items were in all lists my $numlists = @lists; my $all = (1<<$numlists)-1; print "\nThere were $numlists lists.\n\n"; for my $item (sort {$a<=>$b} keys %tally) { if ($numlists == grep { $_ } @{$tally{$item}}) { print "Item $item in all lists:"; for my $idxlist (0 .. $#lists) { print " list $idxlist: ", join(", ", @{$tally{$item}[$id +xlist]}), "\n"; } print "\n"; } $ perl lists_2.pl LIST 0: 5, 6, 9, 13, 3, 0, 1, 2, 5, 5, 10, 14, 15, 8, 4 LIST 1: 11, 5, 2, 12, 3, 4, 9, 13, 14, 14, 9, 15, 5, 2, 6 LIST 2: 6, 0, 7, 12, 11, 2, 13, 8, 14, 8, 13, 5, 0, 1, 7 LIST 3: 3, 14, 14, 11, 7, 10, 12, 8, 6, 15, 5, 6, 14, 6, 11 There were 4 lists. Item 5 in all lists: list 0: 0, 8, 9 list 1: 1, 12 list 2: 11 list 3: 10 Item 6 in all lists: list 0: 1 list 1: 14 list 2: 0 list 3: 8, 11, 13 Item 14 in all lists: list 0: 11 list 1: 8, 9 list 2: 8 list 3: 1, 2, 12

    ...roboticus

    When your only tool is a hammer, all problems look like your thumb.

      Wow, thanks. I was staring at a blank screen needing inspiration, but that should be perfect.

Re: How to compare 3 lists
by grizzley (Chaplain) on Oct 25, 2012 at 10:22 UTC

    Should the date-of-creation be the same in all three lists or not?

    Is snapshot1 named 'snapshot1' in all lists? Then just read one list after another and for every list search for word 'snapshot1', 'snapshot2', etc. without even comparing lists.

      Yes, zfs allows you to use a tunnel (we use mbuffer) to send snapshots between systems meaning you can create an independent mirror but without the performance penalty of synchronous systems like drbd (the downside is that you may use changes)..

        Sorry, I have no idea about zfs and therefore no idea if this is 'yes' for timestamp or 'yes' for second question.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1000803]
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others meditating upon the Monastery: (5)
As of 2024-04-19 07:32 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found