Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

Using mod_log_post output to unit test Apache httpd conf changes.

by jffry (Hermit)
on Jan 05, 2010 at 19:14 UTC ( #815803=CUFP: print w/ replies, xml ) Need Help??

I needed to capture the data from HTTP POST requests on our Apache httpd server. I used mod_log_post. I needed this data so I could to use it as a basis for a quickie sniff test against changes to our Apache httpd server I was planning.

I wrote a script that parsed the post_log output created by mod_log_post and perform the exact same HTTP POST request on whatever host I pointed it at.

This is nothing fancy. Just a quick hack to get a unit test going. For example, if I wanted a real stressor, I should probably make it multi-threaded, and so on.

WARNING:You should read my comments on how I decide what is POST data and what is not. I don't know the HTTP spec well enough, and your POST data might not obey these rules.

Here is the whole thing, screen scraped right from my console.

#!/usr/bin/perl -w # ** Design Note ** # # Right now this script is data-driven. Thus, I could have used the ' +-an' # flags to Perl. This turns Perl into an awk-style script. However, +I may # want to make this non-data-driven later, so it isn't worth it. use strict; use warnings; # Need these 3 for do_test() that performs the actual HTTP request. use HTTP::Request; use HTTP::Headers; use LWP::UserAgent; # Process command line options/flags. use Getopt::Std; my %opt; getopts('h:pv', \%opt); # Debug logging on or off. my $verbose = 0; if (defined $opt{'v'}) { $verbose = 1; } # Pretend on or off. my $pretend = 0; if (defined $opt{'p'}) { $pretend = 1; } # Which host to test. # This is a global. It doesn't have to be. Might refactor later, but + now it is OK. my $host = 'localhost:80'; if (defined $opt{'h'}) { $host = $opt{'h'}; } my $flag = 0; # When true, current line is POST data. my $uri; # Holds URI string. my $data; # Holds POST data for a single URI. while (<>) { # If the line starts with "POST" this must be the URI line. if (/^POST/) { print STDERR 'DEBUG: /^POST/ re line = ' . $_ . "\n" i +f $verbose; $uri = (split /\s+/, $_)[1]; print "$uri\n\n" if $pretend; next; } # A number by itself signals the beginning of the HTTP body, w +hich contains the POST data. if (/^\d+$/ && $flag == 0) { print STDERR 'DEBUG: /^\d+$/ re turned flag on' . "\n" + if $verbose; $flag = 1; next; } if ($flag) { $data .= $_; print if $pretend; } # A blank line by itself signals the end of the POST data. if (/^\s*$/ && $flag == 1) { print STDERR 'DEBUG: /^\s*$/ re turned flag off' . "\n +" if $verbose; print "\n" if ($pretend); do_test($uri, $data); $data = ''; $uri = ''; $flag = 0; next; } } sub do_test { my ($uri, $data) = @_; # Uses the $host global. If you're going to refactor and pass + 3 args to # this sub, then please convert to named args (pass a hash ref +). my $head = HTTP::Headers->new( Content_Type => 'txt/xml', ); $uri = 'http://' . $host . $uri; my $req = HTTP::Request->new( 'POST', $uri, $head, $data); if ($pretend) { print STDERR "DEBUG: Pretend flag set. Not really doin +g HTTP request.\n" if $verbose; print $req->as_string() . "\n\n"; } else { my $ua = LWP::UserAgent->new; my $resp; $resp = $ua->request($req); # This actually performs t +he HTTP request. print STDERR $resp->status_line . "\n" if $verbose; } } __END__ =pod =head1 Usage ./test_from_postlog.pl [options] post_log_files ... =head1 Options =head2 -h hostname:port To target a system other than localhost:80. =head2 -p Pretend/print only. Don't test, just print relevant lines of post_log + file. =head2 -v Turns on verbose logging to stderr. =cut

Here is what a segment of post_log output looks like.

==f3731b40============================== Request: bob 10.0.1.5 - - [29/Dec/2009:18:11:59 --0500] "POST /our/uri +/is.here HTT P/1.0" 200 218 "-" "-" SzqMvwoEVDQAAHGZBgIAAAAB "-" Handler: jakarta-servlet ---------------------------------------- POST /our/uri/is.here HTTP/1.0 Host: bob:80 Content-Type: text/xml Content-Length: 149 149 <?xml version='1.0' encoding='ISO-8859-1'?><BobSaysSomething><SomeData +Date>12/29/2009</SomeDataDate></BobSaysSomething> HTTP/1.1 200 OK Content-Length: 218 Connection: close Content-Type: text/xml --f3731b40--

Comment on Using mod_log_post output to unit test Apache httpd conf changes.
Select or Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: CUFP [id://815803]
Approved by zwon
Front-paged by Arunbear
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (9)
As of 2015-07-02 01:38 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (25 votes), past polls