Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Managing your IP space with Perl

by fokat (Deacon)
on Aug 15, 2002 at 19:02 UTC ( #190497=perltutorial: print w/ replies, xml ) Need Help??

What this tutorial is all about...

Some of us, monks who love Perl, also have to deal with the complexities of IP addresses, subnets and such. A while ago, I wrote NetAddr::IP to help me work out tedious tasks such as finding out which addresses fell within a certain subnet or allocating IP space to network devices.

This tutorial discusses many common tasks along with solutions using NetAddr::IP. Since Perl lacks a native type to represent either an IP address or an IP subnet, I feel this module has been quite helpful for fellow monks who like me, need to work in this area.

Note however that neither the module itself nor this tutorials are intended as replacements to your knowledge about how to work with chunks of IP space. The module was written as a tool to help with the boring tasks (after all, we're plentiful with good laziness, aren't we?) and this tutorial, was written to help answer the most common questions I get. Both the module and this tutorial expect you to be fluent in basic networking and somewhat fluent in Perl. You should not writing Perl code to manage your subnetting otherwise.


Specifying an IP Address or a subnet

A NetAddr::IP object represents a subnet. This involves storing an IP address within the subnet along with the subnet's netmask. Of course, using a host netmask (/32 or in decimal notation, 255.255.255.255) allows for the specification of a single IP address.

You can create a NetAddr::IP object with an incantation like the following:

use NetAddr::IP; my $ip = new NetAddr::IP '127.0.0.1';

which will create an object representing the 'address' 127.0.0.1 or the 'subnet' 127.0.0.1/32.

Creating a subnet is equally easy. Just specify the address and netmask in almost any common notation, as in the following examples:

use NetAddr::IP; my $loopback = new NetAddr::IP '127.0.0.1', '255.0.0.0'; my $rfc1918 = new NetAddr::IP '10.0.0.0/8'; my $another = new NetAddr::IP '1.2.0.0/255.255.0.0'; my $loopback2 = new NetAddr::IP 'loopback';

The following is a list of the acceptable arguments to ->new() and their meanings:

  • ->new('broadcast')

    Equivalent to the address 255.255.255.255/32 which is often used to denote a broadcast address.

  • ->new('default')

    Synonym to the address 0.0.0.0/0 which is universally used to represent a default route. This subnet is guaranteed to ->contains() any other subnet. More on that later.

    For the benefit of many Cisco users out there, any is considered a synonym of default.

  • ->new('loopback')

    The same as the address 127.0.0.1/8 which is the standard loopback address.

  • ->new('10.10.10.10') or ->new('foo.bar.com')

    This represents a single host. When no netmask is supplied, a netmask of /32 is assumed. When supplying a name, the host name will be looked up using gethostbyname(), which will in turn use whatever name resolution is configured in your system to obtain the IP address associated with the supplied name.

  • ->new('10.10.1')

    An ancient notation that allows the middle zeroes to be skipped. The example is equivalent to ->new('10.10.0.1').

  • ->new('10.10.1.')

    Note the trailing dot. This format allows the omission of the netmask for classful subnets. The example is equivalent to ->new('10.10.1.0/24').

  • ->new('10.10.10.0 - 10.10.10.255')

    This is also known as range notation. Both ends of an address range are specified. Note that this notation is only supported if the specified subnet can be represented in valid CIDR notation.

  • ->new('10.10.10.0-255')

    This notation is a shorthand for the range notation discussed above. It provides for the specification of an address range where both of its ends share the first octets. This notation is only supported when the specified range of hosts defined a proper CIDR subnet.

  • ->new(1024)

    Whenever the address is specified as a numeric value greater than 255, it is assumed to contain an IP address encoded as an unsigned int.

  • ->new() with two arguments

    Whenever two arguments are specified to ->new(), the first is always going to be interpreted as the IP address and the second will always be the netmask, in any of the formats discussed so far.

    Netmasks can be specified in dotted-quad notation, as the number of one-bits or as the equivalent unsigned int. Also, special keywords such as broadcast, default or host can be used as netmasks.

The semantics and notations depicted above, are supposed to comply strictly with the DWIM approach which is so popular with Perl. The general idea is that you should be able to stick almost anything resembling an IP address or a subnet specification into the ->new() method to get an equivalent object. However, if you can find another notation that is not included in the above list, please by all means let me know.


Simple operations with subnets

There is a number of operations that have been simplified along the different versions of the module. The current version, as of this writing, provides support for the following operations:

  • Scalarization

    A NetAddr::IP object will become its CIDR representation whenever a scalar representation for it is required. For instance, you can very well do something like print "My object contains $ip\n";.

  • Numerical comparison

    Two objects can be compared using any of the numerical comparison operators. Only the address part of the subnet is compared. The netmask is ignored in the comparison.

  • Increments and decrements

    Adding or substracting a scalar from an object will change the address in the subnet, but always keeping it within the subnet. This is very useful to write loops, like the following:

    use NetAddr::IP; my $ip = new NetAddr::IP('10.0.0.0/30'); while ($ip < $ip->broadcast) { print "ip = $ip\n"; $ip ++; }

    which will produce the following output:

    ip = 10.0.0.0/30 ip = 10.0.0.1/30 ip = 10.0.0.2/30
  • List expansion

    When required, a NetAddr::IP will expand automatically to a list containing all the addresses within a subnet, conveniently leaving the subnet and the broadcast addresses out. The following code shows this:

    use NetAddr::IP; my $ip = new NetAddr::IP('10.0.0.0/30'); print join(' ', @$ip), "\n";

    And the output would be

    10.0.0.1/32 10.0.0.2/32

Common (and not so common) tasks

Below I will try to provide an example for each major feature of NetAddr::IP, along with a description of what is being done, where appropiate.

Optimising the address space

This is one of the reason for writing NetAddr::IP in the first place. Let's say you have a few chunks of IP space and you want to find the optimum CIDR representation for them. By optimum, I mean the least amount of CIDR subnets that exactly represent the given IP address space. The code below is an example of this:

use NetAddr::IP; push @addresses, NetAddr::IP->new($_) for &lt;DATA>; print join(", ", NetAddr::IP::compact(@addresses)), "\n"; __DATA__ 10.0.0.0/18 10.0.64.0/18 10.0.192.0/18 10.0.160.0/19

Which will, of course, output 10.0.0.0/17, 10.0.160.0/19, 10.0.192.0/18. Let's see how this is done...

First, the line starting with push ... creates a list of objects representing all the subnets read in via the <DATA> filehandle. There should be no surprises here.

Then, we call NetAddr::IP::compact with the list of subnets build earlier. This function accepts a list of subnets as its input (actually, an array of objects). It processes them internally and outputs another array of objects, as summarized as possible.

Using compact() as in the example is fine when you're dealing with a few subnets or are writing a throw-away one-liner. If you think your script will be handling more than a few tens of subnets, you might find compactref() useful. It works exactly as shown before, but takes (and returns) references to arrays. I've seen 10x speed improvements when working with huge lists of subnets.

Something that gets asked quite frequently is "why not @EXPORT or at least, @EXPORT_OK methods such as compact()?". The answer is that I believe compact() to be a very generic name, for an operation that is not always used. I think fully qualifying it, adds to the mnemonics of what's being done while not polluting the namespace innecesarilly.


Assigning address space

This problem can be tought as the complement to the prior one. Let's say a couple of network segments need to be connected to your network. You can carve slices out of your address space easily, such as in the following code:

use NetAddr::IP; print "My address space contains the following /24s:\n", join("\n", NetAddr::IP->new('10.0.0.0/22')->split(24)), "\n";

Which will divide your precious address space (the one specified in the NetAddr::IP->new()) in subnets with a netmask of 24 bytes. This magic is accomplished by the ->split() method, which takes the number of bits in the mask as its only parameter. It returns a list of subnets contained in the original object.

Again, in situations where the split might return a large number of subnets, you might prefer the use of ->splitref(), which returns a reference to an array instead.

Returning to our example, you might assign a /24 to each new subnet. Ok, perhaps assigning a /24 is not that good an example, as this falls on an octet boundary but trust me, when you have to split a /16 in /20s, to be allocated in chunks of /22s in a network spanning the whole country, it's nice to know your subnetting is well done.


Cisco's wildcard notation (and other dialects)

Those of you who have had to write an ACL in a Cisco router, know about the joys of this peculiar format in which the netmask works the opposite of what custom says.

An easy way to convert between traditional notation and Cisco's wildcard notation, is to use the eloquently named ->wildcard() method, as this example shows:

use NetAddr::IP; print join(' ', NetAddr::IP->new('10.0.0.0/25')->wildcard());

As you might have guessed, ->wildcard() returns an array whose first element is the address and its second element is the netmask, in wildcard notation. If scalar context is forced using scalar, only the netmask will be returned, as this is most likely what you want.

In case you wonder, the example outputs 10.0.0.0 0.0.0.127.

Just for the record, below is a number of outputs from different methods for the above example:

  • Range (The ->range() method)

    Outputs 10.0.0.0 - 10.0.0.127. Note that this range goes from the network address to the broadcast address.

  • CIDR notation (The ->cidr() method)

    As expected, it outputs 10.0.0.0/25.

  • Prefix notation (The ->prefix() method)

    Similar to ->range(), this method produces 10.0.0.1-127. However, note that the first address is not the network address but the first host address.

  • n-Prefix notation (The ->nprefix() method)

    Produces 10.0.0.1-126. Note how the broadcast address is not within the range.

  • Numeric (The ->numeric() method)

    In scalar context, produces and unsigned int that represents the address in the subnet. In array context, both the address and netmask are returned. For the example, the array output is (167772160, 4294967168). This is very useful when serializing the object for storage. You can pass those two numbers back to ->new() and get your object back.

  • Just the IP address (The ->addr() method)
  • Just the netmask as a dotted quad (The ->mask() method)
  • The length in bits of the netmask (The ->masklen() method)

Matching against your address space

Let's say you have a log full of IP addresses and you want to know which ones belong to your IP space. A simple way to achieve this is shown below:

use NetAddr::IP; my $space = new NetAddr::IP->new('10.128.0.0/17'); for my $ip (map { new NetAddr::IP->new($_) } &lt;DATA>) { print $ip, "\n" if $space->contains($ip); } __DATA__ 172.16.1.1 172.16.1.5 172.16.1.11 172.16.1.10 172.16.1.9 172.16.1.3 172.16.1.2 172.16.1.7 172.16.1.4 172.16.1.1 10.128.0.1 10.128.0.12 10.128.0.13 10.128.0.41 10.128.0.17 10.128.0.19

This code will output only the addresses belonging to your address space, represented by $space. The only interesting thing here is the use of the ->contains() method. As used in our example, it returns a true value if $ip is completely contained within the $space subnet.

Alternatively, the condition could have been written as $ip->within($space). Remember that TIMTOWTDI.


Walking through the network without leaving the office

Some of the nicest features of NetAddr::IP can be better put to use when you want to perform actions with your address space. Some of them are discussed below.

One of the most efficient ways to walk your address space is building a for loop, as this example shows:

use NetAddr::IP; push @space, new NetAddr::IP->new($_) for &lt;DATA>; for my $netblock (NetAddr::IP::compact @space) { for (my $ip = $netblock->first; $ip <= $netblock->last; $ip++) { # Do something with $ip } } __DATA__ 10.0.0.0/16 172.16.0.0/24

The nicest thing about this way of walking your IP space, is that even if you are lucky enough to have lots of it, you won't eat all your memory by generating a huge list of objects. In this example, only one object is created in every iteration of the loop.

Everything up to the inner loop should be pretty clear by now, so we just ignore it. Since a couple of new friends were introduced in the inner loop of our example, an explanation is in order.

This C-like for loop uses the ->first() function to find the first subnet address. The first subnet address is defined as that having all of its host bits but the rightmost set to zero and the rightmost, set to one.

We then use the numerical comparison discussed earlier to see if the value of $ip is less than or equal to whatever ->last() returns. ->last() returns an address with all of its host bits set to one but the rightmost. If this condition holds, we execute the loop and post-increment $ip to get the next IP address in the subnet.

I started the discussion on this topic with the approach that insures less wasted resources. However, in the purest Perl tradition, this is not the only way to do it. There's another way, reserved for the true lazy (or those with memory to burn, but we all know you never have enough memory, right?).

This other way is invoked with the ->hostenum() or the ->hostenumref() methods. They return either an array or a reference to an array respectively, containing one object for each host address in the subnet. Note that only valid host addresses will be returned (as objects) since the network and broadcast addresses are seldom useful.

With no further preamble, I introduce an example that kids shouldn't attempt at home, or at least in production code. (If you find this warning superfluous, try adding 64.0.0.0/8 to the __DATA__ section and see if your machine chews through it all).

use NetAddr::IP; push @space, new NetAddr::IP->new($_) for &lt;DATA>; for my $ip (map { $_->hostenum } NetAddr::IP::compact @space) { # Do something with $ip } __DATA__ 10.0.0.0/16 172.16.0.0/24

If you really have enough memory, you'll see that each host address in your IP space is generated into a huge array. This is much more costly (read, slow) than the approach presented earlier, but provides for more compact one-liners or quickies.


Finding out how big is your network

Have you wondered just how many IP addresses can you use in your current subnet plan? If the answer to this (or to a similar question) is yes, then read on.

There is a method called ->num() that will tell you exactly how many addresses can you use in a given subnet. For the quick observers out there, you can also use something like scalar $subnet->hostenum but this is a really expensive way of doing it.

A more conservative (in resources) approach is depicted below:

use NetAddr::IP; my $hosts = 0; push @space, new NetAddr::IP->new($_) for &lt;DATA>; $hosts += $_->num for @space; print "You have $hosts\n"; __DATA__ 10.0.0.0/16 172.16.0.0/24

Sometimes, you will be surprised at how many addresses are lost by subnetting, but we'll leave that discussion to other tutorials.


Comment on Managing your IP space with Perl
Select or Download Code
Re: Managing your IP space with Perl
by ehdonhon (Curate) on Aug 16, 2002 at 05:04 UTC

    Lots of content, very useful, so I give it a ++.

    You could have also mentioned the existence of other modules that do just about the same thing, such as Net::Netmask and Net::CIDR. A description of when each of them is most useful would be a plus.

Re: Managing your IP space with Perl
by blakem (Monsignor) on Aug 16, 2002 at 09:02 UTC
      FYI,

      Just for kicks, I was going to try running a benchmark of this module on the nm5.in file in Challenge Problem: Merging Network Addresses, and since I'm on WinXP at the moment, I tried using ppm to install this module, and could not install it because it was not in ActiveState's ppd repository (yes, I know if it's pure perl I can just copy it, but I don't want to do that badly enough). When I checked the ppd build status, I saw that it failed on several platforms (Win, Solaris,HP). You may or may not want to address that issue in the next version, if possible. Even if I could run it on the nm5.in file in Dominus' test suite, I see that it would involve creating a rather large array using this method (then again, so did using Net::CIDR). I don't know how 'real world' that file is, so I couldn't say how fair it is to use it in judging your module. Just one issue to keep in mind when solving this sort of problem :-)

      Update: Sorry, meant to reply to the root post. Also just noticed that this whole thread is a couple of years old, just saw it due to a recent post further down.

        >When I checked the ppd build status, I saw that it failed on
        >several platforms (Win, Solaris,HP).
        It didnt seem to fail, just not pass all tests because some other module (Test::Pod) was not installed.... Is this so old that now a new module is used instead? Anyone knows which?
        To install: make a directory called NetAddr in ...perl/site/lib and copy the ip.pm file there as mentioned (this post for completness and because being a newbe it took me some amount of time)
Re: Managing your IP space with Perl
by terryfunk (Sexton) on Nov 18, 2004 at 06:06 UTC
    Nice tutorial...however it would be nice to add a routine that would also sort the ip addresses.
      Nice tutorial...however it would be nice to add a routine that would also sort the ip addresses.

      Well, actually NetAddr::IP objects behave nicely when sorted, so this...

      sort @list_of_netaddr_ip_objects;

      ...Will do as expected. The trick is that the comparison operators used by sort and others, are overloaded too. The overload was done because I needed to sort addresses :)

      Best regards

      -lem, but some call me fokat

Re: Managing your IP space with Perl
by tcc (Novice) on Nov 04, 2005 at 18:46 UTC
    Wow this is nice because I am in the CCNA program and this could prove pretty useful. Nice. I will show my teach.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perltutorial [id://190497]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (7)
As of 2014-09-16 22:55 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (51 votes), past polls