teamassociated has asked for the wisdom of the Perl Monks concerning the following question:

All, I managed to struggle through to get my desired output which is sorting on each array element inside its hash. See out output below. However my goal was/is to alert on any differences after the sort has executed. But I opted to use is_deeply w/no sort and this proved not to work due to how the OS created its v-devices even with manual in order creation. The way I have it now is a push, then I sort using the Schwartzian transform then I print. Is there a better way to push to @{$lsmap{...}} HoA while sorting? I hate to have to populate and sort separately. Thank you!


0x00000010 => Available r2wpar_r1.610b24bfc20149ff17976c9c63f1a261 Available r2wpar_w1.2458bd43d1f3116046180e5e0e361ca7 0x0000001e => Available sapepp00_r1.2915b2239121c2eac69518efed0fbbe4 Available sapepp00_r2.88bdab9aa8334093e1bceaf393f8f88f Available sapepp00_e1.c330a9065d0c3243fe13016d5932c146 Available sapepp00_e2.7b1627b996222f9c74ab8393059788a9 Available sapepp00_p1.761becb64095de97b26e35d419ea2bcb 0x00000012 => Available sapepq00_r1.eacbba530a846f2e40315e892fb66532 Available sapepq00_r2.6d869942baf6fe277f2709fe604bea6d Available sapepq00_e1.0b6ed1a6d9bf9298d7465255b3ab33b4 Available sapepq00_e2.8e55566cd9b5fb35623cd958b4a317d4 Available sapepq00_p1.6beaabc01ab495cf83a138a1ba954d55 __END_OUT__ __RAW_INPUT__ SVSA Physloc Client Pa +rtition ID --------------- -------------------------------------------- --------- +--------- vhost43 U8284.22A.21AAF9V-V1-C33 0x0000000 +8 VTD vtscsi71 Status Available LUN 0x8100000000000000 Backing device twcsapsmmdb1_r1.5c3d6fbe5fa85c487f817a4ae464da0b Physloc Mirrored N/A VTD vtscsi94 Status Available LUN 0x8200000000000000 Backing device twcsapsmmdb1_s1.00dffa2880291e123ca1283ea3f26f2d Physloc Mirrored N/A SVSA Physloc Client Pa +rtition ID --------------- -------------------------------------------- --------- +--------- vhost9 U8284.22A.21AAF9V-V1-C16 0x0000001 +0 VTD vtscsi110 Status Available LUN 0x8100000000000000 Backing device r2wpar_r1.610b24bfc20149ff17976c9c63f1a261 Physloc Mirrored N/A VTD vtscsi111 Status Available LUN 0x8200000000000000 Backing device r2wpar_w1.2458bd43d1f3116046180e5e0e361ca7 Physloc Mirrored N/A __END_RAW__ use strict; use warnings; use File::Find; use Mail::Mailer; use Test::More tests => 2; use vars ('*name', '*dir', '*prune'); *name = *File::Find::name; *dir = *File::Find::dir; *prune = *File::Find::prune; $| = 1; my $regex = qr/\-$|^svsa.*$|^mirrored.*|^physloc.*|^$/i; my (%lsmap15a,$lsm15a,$client15a,$status15a,$bdev15a,$lsmap15a,$lsm15A +); open (LSMAP15A, "+< $lsmap15a") or die $!; while ( $lsm15a = (<LSMAP15A>) ) { next if $lsm15a =~ m/$regex/; ($lsm15A = $lsm15a) =~ s/\-//g; if ( $lsm15a =~ m/vtopt\d+/i ) { my $skip = 3; <LSMAP15A> while $skip-- >0 and not eof LSMAP15A; next; } if ( $lsm15a =~ /^vhost\d+\s+\w+\.\d+\w\.\w+\-\w+\-\w+\s+(0x\w+$)/ +i ) { $client15a = $1; } if ( $lsm15a =~ /^status\s+(\w+$)/i ) { $status15a = $1; push ( @{$lsmap15a{$client15a}}, $status15a ); next; } if ( $lsm15A =~ /^backing\sdevice\s+(\w+_\w+\.\w+$)/i ) { $bdev15a = $1; push ( @{$lsmap15a{$client15a}}, $bdev15a ); next; } } print RLOG "\n"; Test::More->builder->output( $runlog ); Test::More->builder->failure_output( $errlog ); is_deeply(\%lsmap15a, \%lsmap15b, 'vios15a-vios15b-map-compare' ); -s $errlog ? __mailme($contacts, $mesg, $errlog) : sleep 1; ### print w/no sort ### #for my $clientdataA ( sort keys %lsmap15a ) { # print RLOG join ("\n\n", "$clientdataA\t==> #\n\t@{$lsmap15a{$clientdataA}}\n\n"); #} ### print with sort ### foreach( sort { $a->[1] cmp $b->[1] } map { [ $_, $lsmap15a{$_}->[1] ] } keys %lsmap15a) { print join "\n", $_->[0],' => ',@{$lsmap15a{$_->[0]} }; print "\n\n"; }

Replies are listed 'Best First'.
Re: sorted HoA, now what?
by u65 (Chaplain) on Nov 19, 2015 at 23:43 UTC

    Hi, teamassociated, I'm sure a wiser monk than I will chime in eventually, but if I have some time later I will try to tackle your quest. Be forewarned: I usually use a different approach to handling such data files and will probably rewrite the code in my own way, and the first thing I will do is make it not so fearsome looking (also known as 'ugly code'). Later...

    Update: Hm, I'm having trouble seeing how your results come from your inputs. I assume the input data is is a shortened version of that that produced the displayed results.

    Update 2: Thanks for more input data! Fom the desired output you show, it looks like all you want is a sorted list of client partition ID's with each ID followed by a sorted list of available backing devices. Am I right, or am I missing something?

    Update 3: Here's a solution I think will work (may need tweaking for your needs). No fanciness, just results. Note that the ID is not sorted numerically but that can be fixed if need be.

    Update 4: I've tidied up the code a bit. Now we need to address what I believe is your fundamental problem: comparing two logs to see what has changed, and the code below only reads in one log file and converts it to a hash. Regarding the issue of keeping a hash sorted, module Tie::Hash::Sorted may be the answer, but I believe that is more work than need. I'll update this later after some more investigation (including finding an old hash comparison function I made a long time ago).

    Update 5: My dim skull is zeroing in on your problem, albeit slowly. Note that I used a HoH instead of a HoA, and the code I was thinking of was based on the Perl Cookbook, Recipe 5.12. But all that is for nought at the moment if your data file can vary so much (if I understand correctly). It might be useful to see first if the code I've provided (updated to track the LUN) gives an accurate read of all your logs. For instance, your latest input indicates the LUN is pertinent but I see nowhere in your original code that you capture it. After we can read a log reliably, then we can tackle finding the differences between two logs.

    Update 6: I suggest you use pseudo code to describe what you want to do. And use the actual names for the items in the logs you need to deal with.

    #!/usr/bin/env perl use strict; use warnings; # ignore lines beginning with: my $regex1 = qr/^\-|^mirrored|^physloc|^svsa|^vtd/i; # beginning of a chunk of interest: my $regex2 = qr/^vhost/i; my $ifil = 'sorted-hoa-input-data.txt'; my %h; # hash to keep client partition IDs open my $fp, '<', $ifil or die "$ifil: $!"; my ($id, $stat, $bd, $lun); while (defined(my $line = <$fp>)) { next if $line =~ m/$regex1/; if ($line =~ m/$regex2/) { my @d = split ' ', $line; $id = $d[2]; } elsif ($line =~ m/^status/i) { my @d = split ' ', $line; $stat = $d[1]; } elsif ($line =~ m/^lun/i) { my @d = split ' ', $line; $lun = $d[1]; } elsif ($line =~ m/^backing\s+device/i) { my @d = split ' ', $line; $bd = $d[2]; if ($stat eq 'Available') { $h{$id}{$bd} = 1; } } } # print the results for my $k (sort keys %h) { print "\n$k\n=>\n"; print "Available\n$_\n" for (sort keys %{$h{$k}}); }


      Hi Tom, yes the input was shortened by a lot. Here you are. Thx so much!!
      # cat lsmap-all.out SVSA Physloc Client Pa +rtition ID --------------- -------------------------------------------- --------- +--------- vhost0 U8284.22A.21AAF8V-V1-C22 0x0000001 +2 VTD vtscsi23 Status Available LUN 0x8100000000000000 Backing device xxxxxxxrqdb1_r1.c24452093bce96e2f48808ed3f7d7eac Physloc Mirrored N/A VTD vtscsi24 Status Available LUN 0x8200000000000000 Backing device xxxxxxxrqdb1_r2.dd39252cef8d45a4c24d59be438ef60d Physloc Mirrored N/A

        See my update 2 in original response.

Re: sorted HoA, now what?
by u65 (Chaplain) on Nov 21, 2015 at 19:22 UTC
      sorry I had a __DATA__ section I had forgot to include the output in the original node called 'current desired output.' FYI I am using the find module to get the latest log files for the comparisons. Comparisons need to happen twice total, once for each pair of VIO servers. Yes your code works just fine to get a solid starting point. But here is my scratch code under __DATA__ using HoA. Thank you!!!
      File::Find::finddepth({'wanted', \&__wanted15B}, '/opt/r2configs/vios1 +5b'); sub __wanted15B { use strict 'refs'; my($dev, $ino, $mode, $nlink, $uid, $gid); $lsmap15b = $name if ($dev, $ino, $mode, $nlink, $uid, $gid) = lst +at $_ and int -M(_) <= $minutes * 60 and /^lsmap-all\.out\z/s; } __DATA__ use Data::Dumper; print Dumper \%lsmap15a; print "\n============================================================= +==\n"; print Dumper \%lsmap15b; #for my $clientdata16A ( sort keys %lsmap16a ) { # print RLOG join ("\n\n", "$clientdata16A\t==>\n\t@{$lsmap16a{$clie +ntdata16A}}\n\n"); #} # #for my $clientdata16B ( sort keys %lsmap16b ) { # print RLOG join ("\n\n", "$clientdata16B\t==>\n\t@{$lsmap16b{$clie +ntdata16B}}\n\n"); #} for my $clientdataA ( sort keys %lsmap15a ) { print RLOG join ("\n\n", "$clientdataA\t==>\n\t@{$lsmap15a{$clientd +ataA}}\n\n"); } #for my $clientdataB ( sort keys %lsmap15b ) { # print RLOG join ("\n\n", "$clientdataB\t==>\n\t@{$lsmap15b{$client +dataB}}\n\n"); #} print "=" x100,"\n\n"; #foreach( # sort { $a->[1] cmp $b->[1] } # map { [ $_, $lsmap15a{$_}->[1] ] } keys %lsmap15a) { # print join "\n", $_->[0],' => ',@{$lsmap15a{$_->[0]} # }; # print "\n\n"; #} foreach( sort { $a->[1] cmp $b->[1] } map { [ $_, $lsmap15a{$_}->[1] ] } keys %lsmap15a) { print join "\n", $_->[0],' => ',@{$lsmap15a{$_->[0]} }; print "\n\n"; }
Re: sorted HoA, now what?
by u65 (Chaplain) on Nov 21, 2015 at 13:57 UTC
Re: sorted HoA, now what?
by u65 (Chaplain) on Nov 24, 2015 at 12:28 UTC