Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change

How Perl saved my night...

by bronto (Priest)
on Feb 19, 2004 at 17:29 UTC ( #330275=CUFP: print w/replies, xml ) Need Help??

This week I am on on-call service.

A few days ago I received an e-mail from the networking group stating that a network reconfiguration will take place tomorrow morning, between 2:00AM and 7:00AM, and the presence of the on-call engineer (me!) was required to check that all our systems that offer services on the Internet will work correctly after the reconfiguration.

I took the networks list involved in the reconfiguration, and passed it to my colleagues asking who managed services on them and how their functionality should be checked (it's not a long time I work here, some things are still unknown to me). It came out that many of those services could be tested by simply trying an HTTP/HTTPS connection, or some DNS queries, while others have to be checked by the network staff directly. So, if network staff could check those connections and queries in a simple manner, I could probabily go to bed instead!

Mmmmhhh... this looks like a good job for Perl! But to make them run a script correctly I had to make sure that all the needed modules were there, the same way that happens when you install a new module...

I decided to create a fake module distribution, and to use the make test procedure to do the tests. Here we go:

h2xs -AX -b 5.6.0 --use-new-tests --skip-exporter Local::Network::Reconfig

et voilà! Let's edit the makefile:

use 5.006; use ExtUtils::MakeMaker; # See lib/ExtUtils/ for details of how to influence # the contents of the Makefile that is written. WriteMakefile( NAME => 'Local::Network::Reconfig', VERSION_FROM => 'lib/Local/Network/', # finds $VER +SION PREREQ_PM => {'Test::More' => 0, 'LWP::UserAgent' => 0, 'Net::DNS' => 0, 'Crypt::SSLeay' => 0, }, ($] >= 5.005 ? ## Add these new keywords supported since 5.005 (ABSTRACT_FROM => 'lib/Local/Network/', # retrieve a +bstract from module AUTHOR => 'Marco <>') : ()), );

Good. I decided to use the file just to store the $VERSION variable and a minimal POD documentation. I'll omit the source.

And now the big part: the tests. The t/Local-Network-Reconfig.t file:

#use Test::More tests => 1; use Test::More qw(no_plan) ; BEGIN { use_ok('Local::Network::Reconfig') }; use_ok('LWP::UserAgent') ; # HTTP tests my @urls = qw( ) ; my $ua = LWP::UserAgent->new ; ok(defined($ua),"LWP::UserAgent ready") ; $ua->timeout(10) ; $ua->protocols_allowed( [ 'http', 'https'] ); foreach my $url (@urls) { my $response = $ua->get($url) ; ok($response->is_success,"GET $url") ; } my @authenticated_services = qw( ) ; foreach my $url (@authenticated_services) { my $response = $ua->get($url) ; is($response->status_line,'401 Access Denied', "Access to $url is authenticated") ; } # DNS tests use_ok('Net::DNS') ; my @nameservers = qw( ; my @queries = qw( ; foreach my $server (@nameservers) { my $resolver = Net::DNS::Resolver->new( nameservers => [$server], recurse => 0, ) ; ok(defined($resolver),"Resolver object for $server") ; foreach my $host (@queries) { ok(defined($resolver->query($host)),"Query to $server for $host") +; } }

Ok, so doing perl Makefile.PL, make and make test actually verified that the services were working correctly. In case of problems one could simply run perl -Mblib t/Local-Network-Reconfig.t and read the verbose output of the test script

So, what I missed? Oh, yes! The README file!!!

USAGE + To use this module type the following: + perl Makefile.PL make make test + After "perl Makefile.PL" you should check if you need to install any of the prerequisite modules. + If something in "make test" goes wrong, you should change in the base directory of the distribution (that one where you typed the "perl Makefile.PL" command) and run: + perl -Mblib t/Local-Network-Reconfig.t + to get a deeper insight of what is going wrong DEPENDENCIES + This module requires these other modules and libraries: + Test::More LWP::UserAgent Net::DNS Crypt::SSLeay

Good: I packaged the scripts with make dist, sent the tar.gz file by e-mail to the network staff and showed them how to use it. And they said that the procedure is so simple that they don't need me tonight!!!

So... good night, netstaff; good night, monks. And... good night, Perl


The very nature of Perl to be like natural language--inconsistant and full of dwim and special cases--makes it impossible to know it all without simply memorizing the documentation (which is not complete or totally correct anyway).
--John M. Dlugosz

Replies are listed 'Best First'.
Re: How Perl saved my night...
by flyingmoose (Priest) on Feb 19, 2004 at 23:35 UTC
    I'm all for innovation, but just a suggestion... why not write a regular program, such that it could be dispatched from other shell scripts? The perversion of the 'make' system seems a little weird.

      I definitely would have solved this by making a more regular script, I'm not familiar enough with h2xs and MakeMaker to even conceptualize a solution this way. Nodes like this is one of the reasons that I read PM, they shows me new ways of approaching and solving problems.

      After reading this I feel like nodding sagely, pulling a hood over my head and going back to my room so that I can ponder the mysteries of ExtUtils::MakeMaker. Something about this non-standard use of Perl made me think of this article on DeveloperWorks about making Linux boot faster by using make to manage dependencies within startup scripts.

        I remember that article, and make can be used for dependancy resolution outside of compiling...true....however this cries out for regular script use, IMHO. I am in agreement with you that "things that make me think are good" of course -- this is why I'm here as well. If it was just Q&A it would be a boring place.

        This is getting me on a usability aspect. This scripts seems to act so very unlike every single Unix utility out there, and if written another way, users would never have to deal with the readme at all. This includes help if the arguments are wrong, using things like GetOpt::Long or GetOpt::Std, etc. Thus it becomes (to the user) "that Perl abomination" rather than a cool use of perl. Keep it simple by following convention, and let folks learn it is Perl after the fact.

        My favorite comment from frozen-bubble: "Yes, it's written in Perl, you non-believers..."

        That's my perl design philosophy.

Re: How Perl saved my night...
by graff (Chancellor) on Feb 22, 2004 at 07:40 UTC
    Hmm. I can certainly see the goodness of packaging up a module like this: if your colleagues scattered around the world are all using "perl -MCPAN" and have it set up nicely to follow dependencies, then consistency among all the test locations is assured. And thank you very much for posting such a clear and simple example of packaging a module.

    As for putting the actual network test logic into the "make test" part of the build/install process -- as opposed to making it part of an executable perl script -- I suppose there might be good reasons for that... e.g. if this was a "one-shot" problem and you don't expect to have to run this sort of distributed test again, then you did the right thing by not leaving a stray executable laying around on everyone's system. Likewise, if you are going to have to do this sort of test again, but next time the parameters (host list, protocol list, or whatever) will be different, it might be just as well to treat it as a new "install/test" process, rather than sending just an updated executable (when folks probably haven't used the original for months) and hoping that nobody made unexpected changes to their perl configurations.

      if this was a "one-shot" problem and you don't expect to have to run this sort of distributed test again, then you did the right thing by not leaving a stray executable laying around on everyone's system

      That's exactly the point!

      Ciao and thanks for your comments!


      The very nature of Perl to be like natural language--inconsistant and full of dwim and special cases--makes it impossible to know it all without simply memorizing the documentation (which is not complete or totally correct anyway).
      --John M. Dlugosz

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: CUFP [id://330275]
Approved by BazB
[ambrus]: Corion: I think I parsed a HTTP header from a string with LWP once. You can definitely use that to create a HTTP message too. The problme is
[ambrus]: that if you do that, you'd have to find where each HTTP response ends, which is nontrivial if you want persistent connections (essential for performance if you have small requests).
[Corion]: ambrus: Yes, ideally an API that I feed the incoming data piece by piece and that I can ask "is that response done" and "what should I do next" and "please construct the appropriate redirect for me"
[Corion]: ambrus: Yes, ideally the module would do all that nasty stuff for me and give me a way to ask it what the current situation is
[ambrus]: Corion: you could also consider using some wrapper over the multi interface of curl, I think Net::Curl might be a good one, since implementing enough of what it expects from the event loop might be easier than a full AnyEvent interface.
[ambrus]: Corion: you could also consider using IO::Async and its POE driver and some HTTP module for it, but I don't know if that would solve your problems with AnyEvent+POE
[ambrus]: Corion: wait, you didn't say POE. You said Prima, let me look up what that is.
[ambrus]: Corion: have you considered just writing an AnyEvent integration for that thing?
[ambrus]: Or perhaps pushing schmorp to write one?
[ambrus]: Also, searching for an existing one on CPAN obviously

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (16)
As of 2016-12-07 16:04 GMT
Find Nodes?
    Voting Booth?
    On a regular basis, I'm most likely to spy upon:

    Results (130 votes). Check out past polls.