Re^5: Check if a scalar contains a complete HTTP request

[Marshall]

Ok, I am getting a better idea of your objective. Thanks for the explanation!
I am curious... why WireShark doesn't already do what you want? I did a quick search and found that there is a !HTTP filter (filter out HTTP traffic) and that capturing with that program's filter might get the job done? I suspect that Wireshark is very performant C code. Maybe some Perl program to post process the WireShark output might work? I don't know, but an idea...

[FloydATC]

Wireshark depends on getting all the certificates handed to it in order to decrypt traffic, so that's a completely different game. Very good for inspecting inbound SSL traffic to your own servers, pretty worthless for outbound traffic.

Think of this as a poor man's alternative to one specific feature found in Palo Alto, CheckPoint or F5. Those are the commercial product offerings I'm familar with that does SSL inspection really, really well. Better, obviously, than anything I could hope to achieve with a Perl hack.

I'm using IO::Socket::SSL::Intercept combined with SNI sniffing to clone certificates and act as a proxy between the client and the server. I didn't find this documented anywhere so in case anyone is curious, here's the trick:

# During setup
my $https = IO::Socket::SSL->new(
  Listen        => 128,
  LocalAddr     => '0.0.0.0',
  LocalPort     => 8443,
  ReuseAddr     => 1,
  SSL_startHandshake      => 0, # Don't start SSL negotiation immediat
+ely on accept()
);

# In the main loop
my $client = $https->accept(); # Just a vanilla socket for now really

# Now fork off a child process (or use threads or whatever...)
# ...

# In the child process, negotiate SSL with the client
my $client = IO::Socket::SSL->start_SSL($client,
  SSL_server    => 1,
  SSL_create_ctx_callback => \&ctx_callback, # First callback
);
# Now establish an SSL connection with the server and relay traffic in
+ plaintext.
# ...


# This callback is invoked when a new CTX has been created
sub ctx_callback {
  my $ctx = shift;
  Net::SSLeay::CTX_set_tlsext_servername_callback($ctx, \&sni_callback
+); # Second callback
}

# This callback is invoked during SSL handshake when SNI is known
sub sni_callback {
  my $ssl = shift;
  my $hostname = Net::SSLeay::get_servername($ssl);
  # Either use IO::Socket::SSL::Intercept to clone a certificate/key p
+air
  # for $hostname or re-use one that was cloned previously
  my ($cert, $key) = get_cert_key_files($hostname);
  Net::SSLeay::use_RSAPrivateKey_file($ssl, $key, &Net::SSLeay::FILETY
+PE_PEM);
  Net::SSLeay::use_certificate_file($ssl, $cert, &Net::SSLeay::FILETYP
+E_PEM);
  # From here on in, SSL handshake should complete with our cloned cer
+tificate
  # if the client has been configured to trust our CA certificate. 
  # Some services depend on their own private CA store and can't be tr
+icked.
}

The closest non-commercial product I've found is Squid, but besides being a nightmare to configure, Squid depends on the destination IP to be intact so it can make the server connection. This means DNAT is right out, you have to use either WCCP or special routing to redirect the traffic. Unfortunately, neither one is an option in my environment since the vast majority of traffic must be completely undisturbed.

[noxxi]

With this goal it might be a good idea to have a look at App::HTTP_Proxy_IMP. This is an SSL intercepting proxy written in Perl, using IO::Socket::SSL::Intercept for SSL interception and Net::Inspect::L7::HTTP for extracting the HTTP requests out of the data stream. And it also features a plugin interface where you could add your own traffic inspection and manipulation.

Or you might go to the Python world and have a look at mitmproxy.