Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Re^4: Can Log4Perl integrated with LWP log SSL/TLS handshaking?

by ted.byers (Monk)
on Aug 04, 2014 at 20:08 UTC ( [id://1096196]=note: print w/replies, xml ) Need Help??


in reply to Re^3: Can Log4Perl integrated with LWP log SSL/TLS handshaking?
in thread Can Log4Perl integrated with LWP log SSL/TLS handshaking?

How do I get that version number? I tried running cpan, as root, and in it's help, it does not seem to give an option for viewing the version number of those packages that are installed. I them executed the command to upgrade IO::Socket::SSL, and cpan told me that all packages for it are up to date, if that helps. I would expect that result to depend on what platform I am using (OpenSuse 13.1, if that matters).

I can't get the server certificate, even with openssl s_client (it times out), though that happily gets the server certificate from every other server I need to communicate with using https. The operators of that site have sent me their root CA certificates (in two files, so I have to figure out how to tell LWP's useragent to use both of them, or so they say -- they went so far as to suggest I put them both in Java's keystore, as if that helps when writing Perl).

Here is my package, which is intended to encapsulate the complexities of using LWP with HTTPS:

package REJBlibUA::client; use strict; use warnings; #use Net::SSL (); use English qw(-no_match_vars); use HTTP::Status; use LWP::UserAgent; use LWP::Protocol::https; use HTTP::Request; use HTTP::Request::Common; use HTTP::Response; use Log::Log4perl qw(:easy get_logger); use UNIVERSAL::require; use Encode qw(decode encode); my $log_prefix = "[http client] "; sub new { my ($class, %params) = @_; die "non-existing certificate file $params{ca_cert_file}" if $params{ca_cert_file} && ! -f $params{ca_cert_file}; die "non-existing certificate directory $params{ca_cert_dir}" if $params{ca_cert_dir} && ! -d $params{ca_cert_dir}; my $self = { logger => '', user => $params{user}, password => $params{password}, timeout => $params{timeout} || 180, ssl_set => 0, no_ssl_check => $params{no_ssl_check}, ca_cert_dir => $params{ca_cert_dir}, ca_cert_file => $params{ca_cert_file}, SSL_cert_file => $params{SSL_cert_file}, SSL_key_file => $params{SSL_key_file}, }; bless $self, $class; my $conf = q( log4perl.logger = INFO, FileApp, ScreenApp # log4perl.logger = TRACE, FileApp, ScreenAp +p log4perl.appender.FileApp = Log::Log4perl::Appender:: +File log4perl.appender.FileApp.filename = lwp.log log4perl.appender.FileApp.layout = PatternLayout log4perl.appender.FileApp.layout.ConversionPattern = %d> %m%n log4perl.appender.ScreenApp = Log::Log4perl::Appender +::Screen log4perl.appender.ScreenApp.stderr = 0 log4perl.appender.ScreenApp.layout = PatternLayout log4perl.appender.ScreenApp.layout.ConversionPattern = %d> %m% +n ); # Initialize logging behaviour Log::Log4perl->init( \$conf ); # Log::Log4perl->infiltrate_lwp(); *trace = *INFO; *conns = *DEBUG; *debug = *DEBUG; $self->{'logger'} = get_logger(); # create user agent $self->{ua} = LWP::UserAgent->new( parse_head => 0, # No need to parse HTML keep_alive => 1, requests_redirectable => ['POST', 'GET', 'HEAD'] ); $self->{ua}->ssl_opts(verify_hostname => 1, SSL_verify_mode => 1); if ($params{proxy}) { $self->{ua}->proxy(['http', 'https'], $params{proxy}); } else { $self->{ua}->env_proxy(); } $self->{ua}->timeout($self->{timeout}); return $self; } sub request { my ($self, $request, $file) = @_; # $request is a HTTP::Request object, created with only the URL # $file is a message, normally an XML file my $logger = $self->{logger}; my $url = $request->uri(); my $scheme = $url->scheme(); print "\$url = $url\n\t\$scheme = $scheme\n"; print "\t\$self->{ssl_set} = ",$self->{ssl_set},"\n"; print "\t\$self->{ca_cert_dir} = ",$self->{ca_cert_dir},"\n"; print "\t\$self->{ca_cert_file} = ",$self->{ca_cert_file},"\n"; $self->_setSSLOptions() if $scheme eq 'https' && !$self->{ssl_set} +; my $result = HTTP::Response->new( 500 ); eval { $result = $self->{ua}->request($request, $file); }; # check result first if (!$result->is_success()) { # authentication required if ($result->code() == 401) { if ($self->{user} && $self->{password}) { $logger->debug( $log_prefix . "authentication required, submitting credentials" ); # compute authentication parameters my $header = $result->header('www-authenticate'); my ($realm) = $header =~ /^Basic realm="(.*)"/; my $host = $url->host(); my $port = $url->port() || ($scheme eq 'https' ? 443 : 80); $self->{ua}->credentials( "$host:$port", $realm, $self->{user}, $self->{password} ); # replay request eval { if ($OSNAME eq 'MSWin32' && $scheme eq 'https') { alarm $self->{timeout}; } $result = $self->{ua}->request($request, $file); }; if (!$result->is_success()) { $logger->error( $log_prefix . "authentication required, wrong credentials" ); } } else { # abort $logger->error( $log_prefix . "authentication required, no credentials available +" ); } } else { $logger->error( $log_prefix . "communication error: " . $result->status_line() ); } } return $result; } sub _setSSLOptions { my ($self) = @_; # SSL handling $ENV{HTTPS_DEBUG} = 1; if ($self->{no_ssl_check}) { # LWP 6 default behaviour is to check hostname # Fedora also backported this behaviour change in its LWP5 pack +age, so # just checking on LWP version is not enough $self->{ua}->ssl_opts(verify_hostname => 0, SSL_verify_mode => +0) if $self->{ua}->can('ssl_opts'); } else { # only IO::Socket::SSL can perform full server certificate val +idation, # Net::SSL is only able to check certification authority, and +not # certificate hostname # IO::Socket::SSL->require(); # IO::Socket::SSL->require('debug3'); use IO::Socket::SSL qw(debug3); die "failed to load IO::Socket::SSL, " . "unable to perform SSL certificate validation.\n" . "You can use 'no-ssl-check' option to disable it." if $EVAL_ERROR; # if ($self->{logger}{debug} >= 3) { # $Net::SSLeay::trace = 2; # } print "\t\t\$LWP::VERSION = $LWP::VERSION\n"; if ($LWP::VERSION >= 6) { print "\t\tSetting cert dir and file if available\n"; $self->{ua}->ssl_opts(SSL_ca_file => $self->{ca_cert_file} +) if $self->{ca_cert_file}; $self->{ua}->ssl_opts(SSL_ca_path => $self->{ca_cert_dir}) if $self->{ca_cert_dir}; $self->{ua}->ssl_opts(SSL_cert_file => $self->{SSL_cert_fi +le}) if $self->{SSL_cert_file}; $self->{ua}->ssl_opts(SSL_key_file => $self->{SSL_key_file +}) if $self->{SSL_key_file}; } } $self->{ssl_set} = 1; } 1;

And here is the calling code, used to test it:

#!/usr/bin/perl use HTTP::Request; use HTTP::Response; use lib './Work'; use REJBlibUA::client; my $method = "POST"; my @requests = (HTTP::Request->new( $method,'https://www.google.ca'), HTTP::Request->new( $method,'https://gremlin.site/cgi- +bin/printenv.pl'), HTTP::Request->new( $method,'https://byerspublishing.com'), HTTP::Request->new( $method,'https://195.160.170.115:8443/soap +proxy/PaymentServer')); my $cnt = 0; foreach my $r (@requests) { my %rp; if ($cnt == 1) { $rp{'ca_cert_file'} = 'rootCA.pem'; $rp{'ca_cert_dir'} = '.'; $rp{'SSL_cert_file'} = 'client.crt'; $rp{'SSL_key_file'} = 'client.key'; } if ($cnt == 3) { #next; $rp{'ca_cert_file'} = 'ECOMM_old_test.pem'; $rp{'ca_cert_dir'} = '.'; $rp{'SSL_cert_file'} = 'OnsoftArch_test.pem'; $rp{'SSL_key_file'} = 'onsoftarch_test.key'; } my $c = REJBlibUA::client->new(%rp); my $resp = $c->request($r); if ($resp->is_success) { print $resp->decoded_content; } else { print STDERR $resp->status_line, "\n"; } $cnt += 1; }

https://gremlin.site/cgi-bin/printenv.pl works perfectly. https://byerspublishing.com fails because it uses only the default certificate Apache comes with (until I buy a proper certificate for it), and thus the server certificate is not trusted. https://195.160.170.115:8443/soapproxy/PaymentServer is the real problem, but it is accessable only through a VPN to the operator's site. I can ping that server, but openssl times out when I try to get the server's certificate. I suspect their server is not sending it's certificate, for whatever reason. But then, I don't know the sequence of events that must happen when the handshaking is to involve both client and server side certificate validation. Is the client supposed to send it's certificate first, and is it supposed to send the certificate for the CA that signed it (even if the server already has that CA certificate), before the server sends it's certificate, or after?

I suppose the fact the problem server is accessable only through a VPN makes diagnosing the problem, and that I can't seem to get it's certificate, makes diagnosing the problem more challenging.

How do I use $IO::Socket::SSL::DEBUG?

Thanks

Ted

Replies are listed 'Best First'.
Re^5: Can Log4Perl integrated with LWP log SSL/TLS handshaking?
by noxxi (Pilgrim) on Aug 05, 2014 at 19:42 UTC
    > How do I get that version number?
    This works the same as with other perl modules, just print $IO::Socket::SSL::VERSION. But according to your description you must have the newest version, that is 1.997.

    > I can't get the server certificate, even with openssl s_client (it times out), though that happily gets the server certificate from every other server I need to communicate with using https.
    If s_client cannot connect then there is probably no valid SSL server at the other end. But, it might also be a buggy server or a server with a bad SSL accelerator in front. Try to restrict the protocol in s_client with the -ssl3 option and see if this helps.

      Thanks

      I finally did solve the problem with connecting to that server using openssl s_client. I can now reliably connect using that

      The problem that remains is that although I provide the same information to my client, written in Perl, the server fails to send it's certificate. This is a puzzle since I have no problem using the same code when connecting to the servers I control, which are set up to require client side certificates also. In those cases, both certificates are exchanged and verified successfully, and the requested data retrieved. Is there any chance that openssl s_client does anything that the LWP user agent, using IO::SOCKET::SSL doesn't do?

      Thanks

      Ted

        > I finally did solve the problem with connecting to that server using openssl s_client...
        How did you solve the problem with s_client? Which options did you use for IO::Socket::SSL when you tried to make it work the same way your s_client setup works?

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1096196]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others rifling through the Monastery: (4)
As of 2024-03-28 18:01 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found