http://www.perlmonks.org?node_id=1148186


in reply to sorted HoA, now what?

Hi, teamassociated, I'm sure a wiser monk than I will chime in eventually, but if I have some time later I will try to tackle your quest. Be forewarned: I usually use a different approach to handling such data files and will probably rewrite the code in my own way, and the first thing I will do is make it not so fearsome looking (also known as 'ugly code'). Later...

Update: Hm, I'm having trouble seeing how your results come from your inputs. I assume the input data is is a shortened version of that that produced the displayed results.

Update 2: Thanks for more input data! Fom the desired output you show, it looks like all you want is a sorted list of client partition ID's with each ID followed by a sorted list of available backing devices. Am I right, or am I missing something?

Update 3: Here's a solution I think will work (may need tweaking for your needs). No fanciness, just results. Note that the ID is not sorted numerically but that can be fixed if need be.

Update 4: I've tidied up the code a bit. Now we need to address what I believe is your fundamental problem: comparing two logs to see what has changed, and the code below only reads in one log file and converts it to a hash. Regarding the issue of keeping a hash sorted, module Tie::Hash::Sorted may be the answer, but I believe that is more work than need. I'll update this later after some more investigation (including finding an old hash comparison function I made a long time ago).

Update 5: My dim skull is zeroing in on your problem, albeit slowly. Note that I used a HoH instead of a HoA, and the code I was thinking of was based on the Perl Cookbook, Recipe 5.12. But all that is for nought at the moment if your data file can vary so much (if I understand correctly). It might be useful to see first if the code I've provided (updated to track the LUN) gives an accurate read of all your logs. For instance, your latest input indicates the LUN is pertinent but I see nowhere in your original code that you capture it. After we can read a log reliably, then we can tackle finding the differences between two logs.

Update 6: I suggest you use pseudo code to describe what you want to do. And use the actual names for the items in the logs you need to deal with.

#!/usr/bin/env perl use strict; use warnings; # ignore lines beginning with: my $regex1 = qr/^\-|^mirrored|^physloc|^svsa|^vtd/i; # beginning of a chunk of interest: my $regex2 = qr/^vhost/i; my $ifil = 'sorted-hoa-input-data.txt'; my %h; # hash to keep client partition IDs open my $fp, '<', $ifil or die "$ifil: $!"; my ($id, $stat, $bd, $lun); while (defined(my $line = <$fp>)) { next if $line =~ m/$regex1/; if ($line =~ m/$regex2/) { my @d = split ' ', $line; $id = $d[2]; } elsif ($line =~ m/^status/i) { my @d = split ' ', $line; $stat = $d[1]; } elsif ($line =~ m/^lun/i) { my @d = split ' ', $line; $lun = $d[1]; } elsif ($line =~ m/^backing\s+device/i) { my @d = split ' ', $line; $bd = $d[2]; if ($stat eq 'Available') { $h{$id}{$bd} = 1; } } } # print the results for my $k (sort keys %h) { print "\n$k\n=>\n"; print "Available\n$_\n" for (sort keys %{$h{$k}}); }

-Tom

Replies are listed 'Best First'.
Re^2: sorted HoA, now what?
by teamassociated (Sexton) on Nov 20, 2015 at 13:32 UTC
    Hi Tom, yes the input was shortened by a lot. Here you are. Thx so much!!
    # cat lsmap-all.out SVSA Physloc Client Pa +rtition ID --------------- -------------------------------------------- --------- +--------- vhost0 U8284.22A.21AAF8V-V1-C22 0x0000001 +2 VTD vtscsi23 Status Available LUN 0x8100000000000000 Backing device xxxxxxxrqdb1_r1.c24452093bce96e2f48808ed3f7d7eac Physloc Mirrored N/A VTD vtscsi24 Status Available LUN 0x8200000000000000 Backing device xxxxxxxrqdb1_r2.dd39252cef8d45a4c24d59be438ef60d Physloc Mirrored N/A

      See my update 2 in original response.

        Hey u65, just a pointer... if you want to reference a post (in this case your own), the canonical (and proper) way to do it is simply grab the id of the post, and tag it like this: [id://1148186]. This produces: Re: sorted HoA, now what?.

        You can change the name of the link by using the | (pipe) separator: [id://1148186|this post], and it'll appear like: this post.

        Using this kind of linking allows monks to click on it without being taken to the site and being (potentially) logged out of PM. Having clickable access to nodes is a very desired effect ;)

        Yes Tom you are correct. Thank you!! I am comparing two log files, fairly large ~2000 lines, but I only show little here. What could be different but never should be is the backing device. I should always have 2 twcsapxrqdb1_x1 in each log file but sometimes the OS puts them where it wants to. The problem with my previous code was when I added a new LUN to the VIOS, it would assign where the last vtscsi ID gap existed. Here is a visual of what it currently does using grep -p. Error warning from Test::More. got archive_d1f but expected archive_r1a.
        # Failed test 'vios16a-vios16b-map-compare' # at stgpool_lun_compare.plx line 204. # Structures begin differing at: # $got->{0x0000000a}[1] = 'archive_d1f.12dd194548df82b3aa92de +094112214c' # $expected->{0x0000000a}[1] = 'archive_r1a.91f0e02e3948dc9f783597 +0a2d79ddaa' *** LOG 1 *** # grep -p archive /opt/r2configs/vios16a/2015/11/19/lsmap-all.out VTD vtscsi12 Status Available LUN 0x8800000000000000 Backing device archive_d1f.12dd194548df82b3aa92de094112214c Physloc Mirrored N/A VTD vtscsi73 Status Available LUN 0x8100000000000000 Backing device archive_r1a.91f0e02e3948dc9f7835970a2d79ddaa Physloc Mirrored N/A VTD vtscsi74 Status Available LUN 0x8200000000000000 Backing device archive_r1b.e812830370e17478eb79d3b7b10b7e9d Physloc Mirrored N/A VTD vtscsi75 Status Available LUN 0x8300000000000000 Backing device archive_d1a.2e1bd9b5a21aee5120c7b4205e967567 Physloc Mirrored N/A VTD vtscsi76 Status Available LUN 0x8400000000000000 Backing device archive_d1b.e24dedc7fd9a4fddfd2464cce6b20a18 Physloc Mirrored N/A VTD vtscsi77 Status Available LUN 0x8500000000000000 Backing device archive_d1c.cb45fea8aed10491e0df5e9578c9a726 Physloc Mirrored N/A VTD vtscsi78 Status Available LUN 0x8600000000000000 Backing device archive_d1d.b4277aa6b6c4b737084913f78104a90c Physloc Mirrored N/A VTD vtscsi79 Status Available LUN 0x8700000000000000 Backing device archive_d1e.2a7a12d38d59d5ae415d64d5869b1144 Physloc Mirrored N/A *** LOG 2 *** root@r2nim01(/usr/local/scripts)$ # grep -p archive /opt/r2configs/vios16b/2015/11/19/lsmap-all.out VTD vtscsi62 Status Available LUN 0x8100000000000000 Backing device archive_r1a.91f0e02e3948dc9f7835970a2d79ddaa Physloc Mirrored N/A VTD vtscsi66 Status Available LUN 0x8800000000000000 Backing device archive_d1f.12dd194548df82b3aa92de094112214c Physloc Mirrored N/A VTD vtscsi69 Status Available LUN 0x8200000000000000 Backing device archive_r1b.e812830370e17478eb79d3b7b10b7e9d Physloc Mirrored N/A VTD vtscsi70 Status Available LUN 0x8300000000000000 Backing device archive_d1a.2e1bd9b5a21aee5120c7b4205e967567 Physloc Mirrored N/A VTD vtscsi71 Status Available LUN 0x8400000000000000 Backing device archive_d1b.e24dedc7fd9a4fddfd2464cce6b20a18 Physloc Mirrored N/A VTD vtscsi72 Status Available LUN 0x8500000000000000 Backing device archive_d1c.cb45fea8aed10491e0df5e9578c9a726 Physloc Mirrored N/A VTD vtscsi73 Status Available LUN 0x8600000000000000 Backing device archive_d1d.b4277aa6b6c4b737084913f78104a90c Physloc Mirrored N/A VTD vtscsi74 Status Available LUN 0x8700000000000000 Backing device archive_d1e.2a7a12d38d59d5ae415d64d5869b1144 Physloc Mirrored N/A