Re^2: need to fix my installation of IO::Socket::SSL, but how?

[ted.byers]

Thanks Rob

I have applied the updates you have provided, although from the output, I was probably already running the latest versions of these packages, as well as Net::HTTP and Net::HTTPS.

I had assumed, obviously, that I had to have openssl in order to have this reliable. However, you have shown me the error of that assumption. But, that leaves the puzzle illustrated by the following output:

C:/Perl64.v.5.14/bin\perl.exe -w c:/Projects/LWP/test.lwp.pl https://w
+ww.google.ca
2012/11/28 20:13:53> Request: POST https://www.google.ca,    User-Agen
+t: libwww-perl/6.02,    Content-Length: 0,    Content-Type: applicati
+on/x-www-form-urlencoded,    (no content)
2012/11/28 20:13:53> Response last request: https://www.google.ca
2012/11/28 20:13:53> Response headers: Content-Type: text/plain,    Cl
+ient-Date: Thu, 29 Nov 2012 01:13:53 GMT,    Client-Warning: Internal
+ response
2012/11/28 20:13:53> Response code: 500
2012/11/28 20:13:53> Response status: Internal Server Error
2012/11/28 20:13:53> Response status: 500 Can't connect to www.google.
+ca:443 (Crypt-SSLeay can't verify hostnames)
Header field names: Content-TypeClient-DateClient-Warning
Previous: undef
2012/11/28 20:13:53> The URL https://www.google.ca was not successfull
+y retrieved.

I had belived that the inability to verify the hostname was due to a problem with my installation WRT openssl. But, if that is incorrect, why does the code fail to verify the hostname? I had even tried each of the following lines of code (separately):

$ua->ssl_opts( SSL_ca_file => Mozilla::CA::SSL_ca_file() );
$ua->ssl_opts( verify_hostnames => 0 );

Of course, $ua is created beforehand using "my $ua = LWP::UserAgent->new;".

Also, I used the statement "$ENV{HTTPS_DEBUG} = 1; " to try to get more detailed debugging information. However, as you can see above, I get very little useful debugging information. How can I get more detailed information about the hand-shaking between my client and the server, and especially, diagnostics as to what can go awry?

Thanks

Ted

[syphilis]

(Crypt-SSLeay can't verify hostnames)

It seems that, somehow, Crypt::SSLeay is getting involved, instead of (or as well as) Net::SSLeay. I suspect you may well find that Net::SSLeay is not being used at all (but I don't know that for sure, and have no experience with Crypt::SSLeay).

One thing I think you could do is to grab the test suite from IO::Socket::SSL and check that those test scripts run ok. For me, some tests are (rightly) skipped, all other tests pass.
If there are problems for you, firstly try with version 1.76 of IO::Socket::SSL (if you're not already using that version) and see if that makes any difference.

It seems there's a connectivity problem ... maybe someone else here has an understanding of precisely what's going wrong. (I don't.)

Cheers,
Rob

[ted.byers]

My first followup question is, where do I find this test suite, and can I just run it simply from the commandline? I am running version 1.76 of IO::Socket::SSL, but it was installed with ActiveState's PPM, and I can't find a testsuite in my Perl directory tree. I found IO-Socket-SSL-1.79 on CPan, but I don't see a testsuite directory in it.

Now, experimenting with this further in an attempt to get more useful, this is weird. I added a 'use' statement to get further debug info in the position shown:

use IO::Socket::SSL qw(debug3);
use Net::SSL ();
use Mozilla::CA;
use LWP::UserAgent;

The "use IO::Socket::SSL qw(debug3);" is new. The inclusion of Net::SSL is there as I had found, by googleing, that it is suppose to increase debugging info (not very effective as far as I can see)

Now, the debug output for getting google using https is as follows:

2012/11/28 22:13:55> Request: GET https://www.google.ca,    User-Agent
+: libwww-perl/6.02,    (no content)
DEBUG: .../IO/Socket/SSL.pm:1645: new ctx 56552016
DEBUG: .../IO/Socket/SSL.pm:363: socket not yet connected
DEBUG: .../IO/Socket/SSL.pm:365: socket connected
DEBUG: .../IO/Socket/SSL.pm:383: ssl handshake not started
DEBUG: .../IO/Socket/SSL.pm:433: set socket to non-blocking to enforce
+ timeout=180
DEBUG: .../IO/Socket/SSL.pm:446: Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:456: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:466: waiting for fd to become ready: SSL w
+ants a read first
DEBUG: .../IO/Socket/SSL.pm:486: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:1633: ok=1 cert=61147744
DEBUG: .../IO/Socket/SSL.pm:1633: ok=1 cert=60140256
DEBUG: .../IO/Socket/SSL.pm:1633: ok=1 cert=60140080
DEBUG: .../IO/Socket/SSL.pm:1193: scheme=www cert=60140080
DEBUG: .../IO/Socket/SSL.pm:1202: identity=www.google.ca cn=*.google.c
+a alt=2 *.google.ca 2 google.ca
DEBUG: .../IO/Socket/SSL.pm:446: Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:456: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:466: waiting for fd to become ready: SSL w
+ants a read first
DEBUG: .../IO/Socket/SSL.pm:486: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:446: Net::SSLeay::connect -> 1
DEBUG: .../IO/Socket/SSL.pm:501: ssl handshake done
DEBUG: .../IO/Socket/SSL.pm:1682: free ctx 56552016 open=56552016
DEBUG: .../IO/Socket/SSL.pm:1687: free ctx 56552016 callback
DEBUG: .../IO/Socket/SSL.pm:1690: OK free ctx 56552016
2012/11/28 22:13:55> Response last request: https://www.google.ca

And the response headers and content of Google's home page follow. So, my second followup question is, why did the explicit inclusion of IO::Socket::SSL have this effect? Why does that one 'use' statement result in Crypt::SSLeay not being involved?

Now, my last question is this. With one of the secure servers I must work with, I get the following:

DEBUG: .../IO/Socket/SSL.pm:1645: new ctx 53640288
DEBUG: .../IO/Socket/SSL.pm:363: socket not yet connected
DEBUG: .../IO/Socket/SSL.pm:365: socket connected
DEBUG: .../IO/Socket/SSL.pm:383: ssl handshake not started
DEBUG: .../IO/Socket/SSL.pm:433: set socket to non-blocking to enforce
+ timeout=180
DEBUG: .../IO/Socket/SSL.pm:446: Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:456: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:466: waiting for fd to become ready: SSL w
+ants a read first
DEBUG: .../IO/Socket/SSL.pm:486: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:446: Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:1320: SSL connect attempt failed with unkn
+own error error:00000000:lib(0):func(0):reason(0)

DEBUG: .../IO/Socket/SSL.pm:452: fatal SSL error: SSL connect attempt 
+failed with unknown error error:00000000:lib(0):func(0):reason(0)
DEBUG: .../IO/Socket/SSL.pm:1320: IO::Socket::INET configuration faile
+d error:00000000:lib(0):func(0):reason(0)

DEBUG: .../IO/Socket/SSL.pm:1682: free ctx 53640288 open=53640288
DEBUG: .../IO/Socket/SSL.pm:1687: free ctx 53640288 callback
DEBUG: .../IO/Socket/SSL.pm:1690: OK free ctx 53640288

My scripts could connect to this site for the past five years, until just the past couple days. The only thing that appears to have changed is that they have new 'extended validation' certificates. Does this warrant a new thread, or is there a hope of getting help diagnosing why this site fails while all others that I have tested succeed?

Thanks

Ted

[Anonymous Monk]

Does this warrant a new thread, or is there a hope of getting help diagnosing why this site fails while all others that I have tested succeed?

http://www.prefetch.net/articles/debuggingssl.html

wget --debug https...

wget http://curl.haxx.se/ca/cacert.pem

wget --ca-certificate=cacert.pem --debug https...

[runrig]

you can set the LWP verify_hostname option to false, or the environment variable mentioned in Net::HTTPS.