Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

How i can extract the part of server report log based on Ipaddress

by perladdict (Chaplain)
on Aug 17, 2010 at 12:08 UTC ( #855478=perlquestion: print w/ replies, xml ) Need Help??
perladdict has asked for the wisdom of the Perl Monks concerning the following question:

Hi monks,
I am trying to extract the web page report log, which will genarate when i clicked any of the link in a web page.
The request and response will be in xml format. There are no of clicks from different ipaddress, i want to see
the request log which genarated from a specific ipaddress. At present i am using perl one liner command which i got from the perl monks.
perl -ane 'print if /<request>/../</request>/' | tail -f logfile
By this i am able to see the xml log from the <request> to </request> tag.It is very difficult for
searching the request log from a specific ip address.The log format is as below
<?xml version="1.0" encoding="UTF-8"?><status>SUCCESS</status> /***********For QA only ****************/ <?xml version="1.0" encoding="UTF-8"?> <request> <visitorID>a4379158-2fb5-494b-a195-ee2cc44df2a2</visitorID> <reportSuiteID>aolmobilewebdev</reportSuiteID> <pageName>wpt: help_pv</pageName> <pageURL>http://mqa.aol.com/portal/default/help.do?icid=ft_help</pageU +RL> <ipAddress>64.12.218.80</ipAddress> <userAgent>ApacheBench/2.0.41-dev</userAgent> <scXmlVer>1.0</scXmlVer> <timezone>-5</timezone> <channel>us.wptportal</channel> <referrer></referrer> <events>event10</events> <server>qawire-ql26.tweb.aol.com</server> <eVar16>ft_help</eVar16> <prop1>wpt: portal</prop1> <prop2>wpt: Help Page</prop2> <prop3>gmt_5</prop3> <prop10>ApacheBench/2.0.41-dev</prop10> <prop12>http://mqa.aol.com/portal/default/help.do</prop12> <prop13>non-authenticated</prop13> <prop14>no referrer</prop14> <prop24>uaid_na</prop24> <prop49>xml api</prop49> </request> <?xml version="1.0" encoding="UTF-8"?><status>SUCCESS</status> /***********For QA only ****************/ <?xml version="1.0" encoding="UTF-8"?> <request> <visitorID>c8b26ed4-3370-4402-abd7-b0519c51d80c</visitorID> <reportSuiteID>aolmobilewebdev</reportSuiteID> <pageName>wpt: help_pv</pageName> <pageURL>http://mqa.aol.com/portal/default/help.do?icid=ft_help</pageU +RL> <ipAddress>10.146.163.43</ipAddress> <userAgent>ApacheBench/2.0.40-dev</userAgent> <scXmlVer>1.0</scXmlVer> <timezone>-5</timezone> <channel>us.wptportal</channel> <referrer></referrer> <events>event10</events> <server>qawire-ql26.tweb.aol.com</server> <eVar16>ft_help</eVar16> <prop1>wpt: portal</prop1> <prop2>wpt: Help Page</prop2> <prop3>gmt_5</prop3> <prop10>ApacheBench/2.0.40-dev</prop10> <prop12>http://mqa.aol.com/portal/default/help.do</prop12> <prop13>non-authenticated</prop13> <prop14>no referrer</prop14> <prop24>uaid_na</prop24> <prop49>xml api</prop49> </request> <?xml version="1.0" encoding="UTF-8"?><status>SUCCESS</status> /***********For QA only ****************/ <?xml version="1.0" encoding="UTF-8"?> <request> <visitorID>420ea1fe-c93e-4209-bee9-e548de8ea1c5</visitorID> <reportSuiteID>aolmobilewebdev</reportSuiteID> <pageName>wpt: help_pv</pageName> <pageURL>http://mqa.aol.com/portal/default/help.do?icid=ft_help</pageU +RL> <ipAddress>64.12.218.64</ipAddress> <userAgent>ApacheBench/2.0.41-dev</userAgent> <scXmlVer>1.0</scXmlVer> <timezone>-5</timezone> <channel>us.wptportal</channel> <referrer></referrer> <events>event10</events> <server>qawire-ql26.tweb.aol.com</server> <eVar16>ft_help</eVar16> <prop1>wpt: portal</prop1> <prop2>wpt: Help Page</prop2> <prop3>gmt_5</prop3> <prop10>ApacheBench/2.0.41-dev</prop10> <prop12>http://mqa.aol.com/portal/default/help.do</prop12> <prop13>non-authenticated</prop13> <prop14>no referrer</prop14> <prop24>uaid_na</prop24> <prop49>xml api</prop49> </request>
I want to see the log </request> to </request> which is from the ipaddress:10.146.163.43

Appreciate your assistance!

Comment on How i can extract the part of server report log based on Ipaddress
Download Code
Re: How i can extract the part of server report log based on Ipaddress
by Anonymous Monk on Aug 17, 2010 at 12:29 UTC
    I don't see why you're using xml to begin with, but why aren't you using an xml parser? XML::Twig or XML::LibXML, or even XML::Simple (i'd probably use XML::Simple to dump the reports into a sqlite database ...)
      To be able to use an XML parser, the OP would have to make the logfile a valid XML (i.e. remove repeated <?xml... lines and add a root element). Then, the desired request could be found by this simple XPath:
      /root/request[ipAddress="10.146.163.43"]
Re: How i can extract the part of server report log based on Ipaddress
by suhailck (Friar) on Aug 17, 2010 at 14:36 UTC
    A sed solution,

    sed -n '/<request>/,/<\/request>/{/<request>/!{H;/<ipAddress>10.146.163.43<\/ipAddress>/{x;p;:a;n;p;/<\/request>/!ba}};/<request>/{h} }' <filename>
      Hi suhailck,
      THanks a lot,i tried my task by your sed code, it is working perfectly.Can you please explain in detail about this command, i gone through the sed online tutorial, still i unable to understand some of the thigs in this code
      in your sed code. If you provide some cool sed tutorial stuff link, it will be a greate help for me in future.
Re: How i can extract the part of server report log based on Ipaddress
by dasgar (Deacon) on Aug 17, 2010 at 14:54 UTC

    My first thought would be to use an XML parser (as previously suggested), but choroba has good point about XML format. Although it's not a one liner, I believe that the untested code below should do the trick.

    use strict; my $ip = '10.146.163.43'; my $logfile = "logfile.log"; my $log; open(LOG,"<",$logfile) || die "Unable to open file '$logfile': $!\n"; { local $/; $log = <LOG>; # slurping file into $log } close(LOG); $log =~ m/(<request>.*?$ip.*?\/request>)/is; print "Log for IP address $ip:\n\n$1\n";

    If you're on a *nix OS, you'll probably need to add the appropriate #! line at the top. I do most of my Perl stuff on Windows and haven't gotten in the habit of adding that line to my code.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://855478]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (6)
As of 2014-09-21 17:31 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (173 votes), past polls