in reply to converting tcpdump output

you also might want to take a look at using ethereal (GUI) and/or tethereal (CLI) as replacements for tcpdump. It's packet selection mechanism is able to analyze HTTP, so you can do your selection of traffic there, for instance this selects all GET and HEAD requests from a live capture on one of my ethernet interface:
tethereal -i eth1 -R'http.request.method == "GET" || http.request.meth +od == "HEAD"'
outputs:
7.518730 66.27.72.141 -> 192.172.226.123 HTTP GET / HTTP/1.1 7.616699 66.27.72.141 -> 192.172.226.123 HTTP GET /HOME/images/nav_1 +_1_HOME.gif HTTP/1.1 7.638719 66.27.72.141 -> 192.172.226.123 HTTP GET /HOME/images/nav_2 +_1_HOME_overview.gif HTTP/1.1 7.641536 66.27.72.141 -> 192.172.226.123 HTTP GET /images/nav_1_2_GL +OBAL_globe.gif HTTP/1.1 7.670158 66.27.72.141 -> 192.172.226.123 HTTP GET /images/nav_2_2_GL +OBAL_globe.gif HTTP/1.1 7.694446 66.27.72.141 -> 192.172.226.123 HTTP GET /HOME/images/nav_3 +_1_HOME_overview.gif HTTP/1.1 7.714087 66.27.72.141 -> 192.172.226.123 HTTP GET /HOME/images/nav_3 +_2_HOME_globe.gif HTTP/1.1
replace -i <iface> with -r <file> to read from file.

Replies are listed 'Best First'.
Re^2: converting tcpdump output
by RnC (Sexton) on May 16, 2005 at 23:12 UTC
    Thanks again!

    Before I posted I was already thinking about installing (t)ethereal to do that. The reason why I prefer tcpdump is the fact that it's default on 90% of GNU/Linux and Unix systems.

    Now consistency worries me. I didn't know that there could be so much variation in the data output format between versions of tcpdump, and specially, that I wouldn't be able to format this output by supplying different parameters. tethereal will end up as my definite solution.
Re^2: converting tcpdump output
by RnC (Sexton) on May 17, 2005 at 00:50 UTC
    Another thing that I missed when running the code you mentioned is that POST payloads are getting stripped as well.

    I managed to see the contents by commenting this line:
    s/^\s+\S+\s+//; # remove initial address ind. #s/\s{2}.*//; # remove trailing dump <-- commented s/\s+//;
    And redirecting the output to a file, then grepping it later. Is there a way of printing it along with the target?

    EDIT: So far, I managed to get the values by modifying the regex line:
    if (/^\S/) { while ($packet=~/(GET|POST|WWW-Authenticate|Authorization|[a-z]+=[a- +z]+).+/g) { ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst)=localti +me(time);
    Basically, I extended the regex to capture a string followed by "=" and then string again.

    But as you can see, it's not a concise solution, since it gets a lot of rubbish as well.
      Ok, I figured it out myself. Here's the complete code.
      #!/usr/bin/perl $|=1; open (STDIN,"sudo /usr/sbin/tcpdump -lnx -s 1024 dst port 80 |"); while (<>) { if (/^\S/) { while ($packet=~/(GET|POST|WWW-Authenticate|Authorizat +ion|Content-Length: \w+\s*).+/g) { ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst)= +localtime(time); printf "%4d/%02d/%02d %02d:%02d:%02d",$year+1900,$mon+ +1,$mday,$hour,$min,$sec; print " - $client -> $host\t$&\n"; } undef $client; undef $host; undef $packet; ($client,$host) = /(\d+\.\d+\.\d+\.\d+\.\d+).+ > (\d+\.\d+\.\d ++\.\d+\.\d+)/ if /P \d+:\d+\((\d+)\)/ && $1 > 0; } next unless $client && $host; s/^\s+\S+\s+//; # remove initial address ind. s/\s{2}.*//; # remove trailing dump s/\s+//; s/([0-9a-f]{2})\s?/chr(hex($1))/eg; tr/\x1F-\x7E\r\n//cd; $packet .= $_; }
      Still not suitable, since it creates a line to just contain POST data, when it would be better if it appended the content to the end of a POST line, or something like that.