Help me decipher code containing map function

[adamZ88]

Hi All,

I have a piece of code that was given to me in class about 4 years ago. I am now starting to use the code heavily. I can simply copy and paste the code into any script requiring it, but I would rather know step by step exactly what the code is doing so that I am better informed to adjust it to other situations. Please let me know if the code is omitting anything that might be considered as default

I am using SNMP to retrieve the ARP table of a Cisco switch. The response is a mac address that comes back packed, so I must unpack it to present it in a human friendly manner.

The end result ends up being something like: abcd.001d.9456. The piece of code is this:

my $val;
        map { $val .= sprintf("%02x",$_) } unpack "CCCCCC", $octetstr;
        $mac=join(".",unpack("a4 a4 a4",$val));

Now, before you tell me to go and do some reading, I have. I am coming to you because I still have some doubts.I.E: I thought that the map function takes in a list(perhaps an array), I do not see the above code having a list passed to it. Unless the packed mac address contained in $octetstr is the list. I am also not quite sure what sprintf("%02x",$_) is ll about. Why is this needed? For better readability of the code, how can I rename the $_ variable. Why do i get an error when I start my code like "map { my $val .=" instead of declaring $val separately. Thank you.

[BrowserUk]

I thought that the map function takes in a list(perhaps an array), I do not see the above code having a list passed to it.

unpack outputs a list. In this case it extracts a list of 6 numbers (ascii values) from the (first) 6 bytes in $octetstr.

It then uses map to iterate over those 6 numbers converting them to 2-digit hex strings and appending them to $val

Before spliting them (the 2-digit hex strings) back apart and then joining them again interspersed with ':'s.

It is an altogether overcomplicated and clumsy piece of code. The whole process can be done with just $mac = join ':', unpack '(H2)*', $octetstr;

Ie. Given:

$octetstr = join '', map chr(), 0xab, 0xcd, 0x00, 0x1d, 0x94, 0x56;;

Instead of:

val = ''; 
map { $val .= sprintf("%02x",$_) } unpack "CCCCCC", $octetstr;;

$mac = join( ":", unpack("a2 a2 a2 a2 a2 a2", $val ) );;

print $mac;;
ab:cd:00:1d:94:56

You could just do:

$mac = join ':', unpack '(H2)*', $octetstr;;

print $mac;;
ab:cd:00:1d:94:56

[adamZ88]

Browser, I very much prefer your method of unpacking the MAC. I have implemented it into my code. Thank you for taking the time to explain. Now I am doubting everything I learned in class :/. I have an additional question, can I compare a packed MAC just as I can compare it after unpacked? like:

if($packed1 eq $packed2){
  print "The packed MACs match\n";
else{
  print "The packed MACs do not match\n";
}

[BrowserUk]

can I compare a packed MAC just as I can compare it after unpacked?

Yes. Before you unpack and format them, your octet strings are just sequences of bytes with binary values in the range 0 -> 255. Ie. strings. Perl's normal string comparison operations (eq, ne, lt, le, gt, ge) will work perfectly on them; and you can use cmp to sort them.

[AnomalousMonk]

I will take the liberty of answering for BrowserUk. I'm not sure what $packed1 and $packed2 are supposed to represent. If $packed1 is the original, raw MAC address and $packed2 is the reformatted address (with bytes represented as hex-digit pairs and with dots thrown in for good measure), then no, they cannot be directly compared:

c:\@Work\Perl\monks>perl -wMstrict -le
"my $octetstr = qq{\x44\x69\x66\x66\x65\x72};
 print qq{raw: '$octetstr'};
 ;;
 my $mac = join '.', unpack '(H4)*', $octetstr;
 print qq{reformatted: '$mac'};
 ;;
 if ($octetstr eq $mac) {
   print qq{'$octetstr' and '$mac' are equal};
   }
 else {
   print qq{'$octetstr' and '$mac' are NOT equal};
   }
"
raw: 'Differ'
reformatted: '4469.6666.6572'
'Differ' and '4469.6666.6572' are NOT equal

(The two strings would have to be converted to a common format for comparison.)

If $packed1 and $packed2 are both reformatted MAC addresses (and they've both been reformatted according to the same scheme), then yes, they can be directly compared for equality.

Incidentally, this is another question that's open to resolution by direct experimentation. Could you not have written a couple of short routines to answer your own question — plus some variations?

[Anonymous Monk]

why (H2)* and not H* ?

[AnomalousMonk]

Because  H* gives you a single string of an unlimited number, necessarily even, of hex characters, but  (H2)* gives you an unlimited number of groups of pairs (or in the case of  (H4)* quartets). The parentheses do grouping in pack/unpack templates. The groups can be quantified. BTW: This is susceptible to experimentation:

c:\@Work\Perl\monks>perl -wMstrict -MData::Dump -le
"my $octetstr = qq{\xab\xcd\x00\x1d\x94\x56};
 ;;
 my @ra = unpack 'H*', $octetstr;
 dd \@ra;
 ;;
 @ra = unpack '(H2)*', $octetstr;
 dd \@ra;
 ;;
 @ra = unpack '(H4)*', $octetstr;
 dd \@ra;
"
["abcd001d9456"]

["ab", "cd", "00", "1d", 94, 56]

["abcd", "001d", 9456]

[afoken]

Because an even number of hex digits (H) is needed.

Alexander

[BrowserUk]

Because bytes (pompously aka. octets) consist of 2 hex digits dec:0 -> 0x00 dec:255 -> 0xff.

[AnomalousMonk]

What BrowserUk said. And if for some reason you really want the address in two-byte groups:

c:\@Work\Perl\monks>perl -wMstrict -le
"my $octetstr = qq{\xab\xcd\x00\x1d\x94\x56};
 ;;
 my $mac = join '.', unpack '(H4)*', $octetstr;
 print qq{'$mac'};
"
'abcd.001d.9456'

Update:

Why do i get an error when I start my code like "map { my $val .=" instead of declaring $val separately.

Because if your code were
    map { my $val .= sprintf("%02x",$_) } unpack "CCCCCC", $octetstr;
    $mac=join(".",unpack("a4 a4 a4",$val));
then the  $val lexical scoped within the map block is entirely local to the map block and does not exist outside of it, so the  $val variable in the
    $mac=join(...,unpack(...,$val));
statement is undeclared and is a syntactic error under strict (if that's the error you're getting) — unless you have an unrelated  $val declared somewhere, which would be a semantic error.

[adamZ88]

Thank you for showing me how to unpack the MAC in additional ways Anomalous, this is very helpful in the networking world!

[Marshall]

I believe that BrowserUk and AnomalousMonk clearly have the best code to use in place of your code from the class: Shorter, more clear, runs faster.

I would like to make a few comments about map. All maps can be written as "foreach" loops. Use "for" instead of "foreach" if you want - these keywords mean exactly the same.

The code that you started with uses what is called a "bare map", a map which does not make use of the potential output list. Most maps are like: @out_array = map{func..}@in_array; In your case, there is no @out_array and the map function works because of side-effects within the map's code. In this case $val is being modified.

I re-wrote the map code into a foreach loop as a demo. This allows you to add print statements to see what the loop is doing. And of course since this is a foreach loop, the $_ variable can be some other name (also see below). Performance between a map and a foreach version should be similar.

I personally only use map for short transformation operations, perhaps making a hash table from a list of values my %hash = map{$_ => 1}@list;. In this hash table case, the code is a "one liner" that makes use of the map's output, i.e. it is not a "bare" map. I don't use map's that do not make use of the left hand assignment. I have seen map functions with 20 lines of code. Again, I would recommend a foreach loop in that situation.

I hope the demo code below helps:

#!/usr/bin/perl
use strict;
use warnings;

my $octetstr = join '', map chr(), 0xab, 0xcd, 0x00, 0x1d, 0x94, 0x56;

my $val;
map { $val .= sprintf("%02x",$_) } unpack "CCCCCC", $octetstr;
my $mac=join(".",unpack("a4 a4 a4",$val));

print "MAC1: $mac\n";

########################
####  Exactly the same thing with foreach instead of map{}
########################

$val='';
foreach my $decimal_value (unpack "CCCCCC", $octetstr)
{ 
   $val .= sprintf("%02x",$decimal_value); # makes 2 digit Hex

   print "Decimal_value: $decimal_value \tval(hex string)=$val\n";
}
my $mac2=join(".",unpack("a4 a4 a4",$val));
print "MAC2: $mac2\n";

# the a4 unpack for $mac2 could be done in other, slower ways
# like with a regex (a very,very slow way, but just a hypothetical)
#

my $mac3 = $val;
$mac3 =~ s/^([0-9a-f]{4})([0-9a-f]{4})([0-9a-f]{4})/$1\.$2\.$3/;
print "MAC3: $mac3\n";

#######################
### BrowserUk, AnomalousMonk ideas with either H2 or H4
### far better!! One step, fast. Clearly the right way
### for this specific problem!
#######################

my $mac4 = join '.', unpack '(H4)*', $octetstr;
print "MAC4: $mac4\n";

__END__
All of these MAC's are the same:

MAC1: abcd.001d.9456
Decimal_value: 171     val(hex string)=ab
Decimal_value: 205     val(hex string)=abcd
Decimal_value: 0     val(hex string)=abcd00
Decimal_value: 29     val(hex string)=abcd001d
Decimal_value: 148     val(hex string)=abcd001d94
Decimal_value: 86     val(hex string)=abcd001d9456
MAC2: abcd.001d.9456
MAC3: abcd.001d.9456
MAC4: abcd.001d.9456

[shmem]

map { $val .= sprintf("%02x",$_) } unpack "CCCCCC", $octetstr;

Nice example for map in void context. I wouldn't transform that into a full foreach loop, but use a statement modifier instead

$val .= sprintf("%02x",$_) for unpack "CCCCCC", $octetstr;

for the same reasons I wrote down elsewhere.

But then - regarding map in void context - all perl statements are executed for their side effect, and the computed value is discarded:

# result from map is discarded
# side effect: $val is populated
map { $val .= sprintf("%02x",$_) } unpack "CCCCCC", $octetstr;

# result from print is discarded (1 if successful)
# side effect: text appears on STDOUT or selected filehandle
print "hello, world!\n";

# result from assignment is 42, again discarded
# side effect: $value is populated
$value = 42;

# result from die is discarded (if any), print not done
# side effect: well...
print die;

[adamZ88]

Your Sample code certainly helps! Thank you!

[haukex]

For better readability of the code, how can I rename the $_ variable.

Well, one way might be with English:

my @x = map {$_*2} 1..3;
# --becomes-->
use English;
my @y = map { $ARG * 2 } 1..3;

But I think the point of map and $_ is to be relatively short, so if you want to improve readability by being a bit more verbose, the general pattern for that is:

my @x = map {sprintf("%02x",$_)} 45..48;
# --becomes-->
my @y;
for (45..48) {
    push @y, sprintf("%02x",$_);
}
# --becomes-->
my @z;
for my $val ( 45 .. 48 ) {
    push @z, sprintf "%02x", $val;
}

I would rather know step by step exactly what the code is doing ... I do not see the above code having a list passed to it.

Here are two debugging techniques to help with that: First, Data::Dump (like Data::Dumper but with a bit nicer output), for example:

use Data::Dump;
dd( unpack "CCCCCC", "\xab\xcd\x00\x1d\x94\x56" );
__END__
(171, 205, 0, 29, 148, 86)

So you see that unpack is returning a list. Second, if you're unsure what context a function is being called in, like localtime(time) in the following example, then to test it you can replace the function call with the whatcontext function I show below, which makes use of wantarray.

my  $q  = localtime(time);
my ($r) = localtime(time);

sub whatcontext {
    print wantarray ? "list" :
        defined wantarray ? "scalar" : "void", "\n";
    return;
}

my  $s  = whatcontext;
my ($t) = whatcontext;
whatcontext;

__END__

scalar
list
void

[adamZ88]

Thank you for explaining that renaming the $_ would defeat the purpose of the map function to an extent. Additional, thank you for providing me for more tools to achieve my goals!