Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re^4: need to fix my installation of IO::Socket::SSL, but how?

by ted.byers (Monk)
on Nov 29, 2012 at 03:47 UTC ( [id://1006153]=note: print w/replies, xml ) Need Help??


in reply to Re^3: need to fix my installation of IO::Socket::SSL, but how?
in thread need to fix my installation of IO::Socket::SSL, but how?

My first followup question is, where do I find this test suite, and can I just run it simply from the commandline? I am running version 1.76 of IO::Socket::SSL, but it was installed with ActiveState's PPM, and I can't find a testsuite in my Perl directory tree. I found IO-Socket-SSL-1.79 on CPan, but I don't see a testsuite directory in it.

Now, experimenting with this further in an attempt to get more useful, this is weird. I added a 'use' statement to get further debug info in the position shown:

use IO::Socket::SSL qw(debug3); use Net::SSL (); use Mozilla::CA; use LWP::UserAgent;

The "use IO::Socket::SSL qw(debug3);" is new. The inclusion of Net::SSL is there as I had found, by googleing, that it is suppose to increase debugging info (not very effective as far as I can see)

Now, the debug output for getting google using https is as follows:

2012/11/28 22:13:55> Request: GET https://www.google.ca, User-Agent +: libwww-perl/6.02, (no content) DEBUG: .../IO/Socket/SSL.pm:1645: new ctx 56552016 DEBUG: .../IO/Socket/SSL.pm:363: socket not yet connected DEBUG: .../IO/Socket/SSL.pm:365: socket connected DEBUG: .../IO/Socket/SSL.pm:383: ssl handshake not started DEBUG: .../IO/Socket/SSL.pm:433: set socket to non-blocking to enforce + timeout=180 DEBUG: .../IO/Socket/SSL.pm:446: Net::SSLeay::connect -> -1 DEBUG: .../IO/Socket/SSL.pm:456: ssl handshake in progress DEBUG: .../IO/Socket/SSL.pm:466: waiting for fd to become ready: SSL w +ants a read first DEBUG: .../IO/Socket/SSL.pm:486: socket ready, retrying connect DEBUG: .../IO/Socket/SSL.pm:1633: ok=1 cert=61147744 DEBUG: .../IO/Socket/SSL.pm:1633: ok=1 cert=60140256 DEBUG: .../IO/Socket/SSL.pm:1633: ok=1 cert=60140080 DEBUG: .../IO/Socket/SSL.pm:1193: scheme=www cert=60140080 DEBUG: .../IO/Socket/SSL.pm:1202: identity=www.google.ca cn=*.google.c +a alt=2 *.google.ca 2 google.ca DEBUG: .../IO/Socket/SSL.pm:446: Net::SSLeay::connect -> -1 DEBUG: .../IO/Socket/SSL.pm:456: ssl handshake in progress DEBUG: .../IO/Socket/SSL.pm:466: waiting for fd to become ready: SSL w +ants a read first DEBUG: .../IO/Socket/SSL.pm:486: socket ready, retrying connect DEBUG: .../IO/Socket/SSL.pm:446: Net::SSLeay::connect -> 1 DEBUG: .../IO/Socket/SSL.pm:501: ssl handshake done DEBUG: .../IO/Socket/SSL.pm:1682: free ctx 56552016 open=56552016 DEBUG: .../IO/Socket/SSL.pm:1687: free ctx 56552016 callback DEBUG: .../IO/Socket/SSL.pm:1690: OK free ctx 56552016 2012/11/28 22:13:55> Response last request: https://www.google.ca

And the response headers and content of Google's home page follow. So, my second followup question is, why did the explicit inclusion of IO::Socket::SSL have this effect? Why does that one 'use' statement result in Crypt::SSLeay not being involved?

Now, my last question is this. With one of the secure servers I must work with, I get the following:

DEBUG: .../IO/Socket/SSL.pm:1645: new ctx 53640288 DEBUG: .../IO/Socket/SSL.pm:363: socket not yet connected DEBUG: .../IO/Socket/SSL.pm:365: socket connected DEBUG: .../IO/Socket/SSL.pm:383: ssl handshake not started DEBUG: .../IO/Socket/SSL.pm:433: set socket to non-blocking to enforce + timeout=180 DEBUG: .../IO/Socket/SSL.pm:446: Net::SSLeay::connect -> -1 DEBUG: .../IO/Socket/SSL.pm:456: ssl handshake in progress DEBUG: .../IO/Socket/SSL.pm:466: waiting for fd to become ready: SSL w +ants a read first DEBUG: .../IO/Socket/SSL.pm:486: socket ready, retrying connect DEBUG: .../IO/Socket/SSL.pm:446: Net::SSLeay::connect -> -1 DEBUG: .../IO/Socket/SSL.pm:1320: SSL connect attempt failed with unkn +own error error:00000000:lib(0):func(0):reason(0) DEBUG: .../IO/Socket/SSL.pm:452: fatal SSL error: SSL connect attempt +failed with unknown error error:00000000:lib(0):func(0):reason(0) DEBUG: .../IO/Socket/SSL.pm:1320: IO::Socket::INET configuration faile +d error:00000000:lib(0):func(0):reason(0) DEBUG: .../IO/Socket/SSL.pm:1682: free ctx 53640288 open=53640288 DEBUG: .../IO/Socket/SSL.pm:1687: free ctx 53640288 callback DEBUG: .../IO/Socket/SSL.pm:1690: OK free ctx 53640288

My scripts could connect to this site for the past five years, until just the past couple days. The only thing that appears to have changed is that they have new 'extended validation' certificates. Does this warrant a new thread, or is there a hope of getting help diagnosing why this site fails while all others that I have tested succeed?

Thanks

Ted

Replies are listed 'Best First'.
Re^5: need to fix my installation of IO::Socket::SSL, but how?
by Anonymous Monk on Nov 29, 2012 at 08:53 UTC

    Does this warrant a new thread, or is there a hope of getting help diagnosing why this site fails while all others that I have tested succeed?

    http://www.prefetch.net/articles/debuggingssl.html

    wget --debug https...

    wget http://curl.haxx.se/ca/cacert.pem

    wget --ca-certificate=cacert.pem --debug https...

      Thanks. That was helpful. It now seems that the problem is really low level, in terms of openssl not being able to talk to the server. Rather, what I get using 'openssl s_client -connect www.theserver.com' suggests that the usual SSL handshaking dies even before the server's certificate is obtained. Here is some of the output it produced:

      Loading 'screen' into random state - done CONNECTED(000000F0) write:errno=10054 --- no peer certificate available --- No client certificate CA names sent --- SSL handshake has read 0 bytes and written 321 bytes --- New, (NONE), Cipher is (NONE) Secure Renegotiation IS NOT supported Compression: NONE Expansion: NONE ---

      And here is the result of the same command when the server is google's home page:

      C:\Work>openssl s_client -connect www.google.com:443 Loading 'screen' into random state - done CONNECTED(000000F0) depth=1 C = ZA, O = Thawte Consulting (Pty) Ltd., CN = Thawte SGC CA verify error:num=20:unable to get local issuer certificate verify return:0 --- Certificate chain 0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=www.google.co +m i:/C=ZA/O=Thawte Consulting (Pty) Ltd./CN=Thawte SGC CA 1 s:/C=ZA/O=Thawte Consulting (Pty) Ltd./CN=Thawte SGC CA i:/C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Au +thority --- Server certificate -----BEGIN CERTIFICATE----- MIIDITCCAoqgAwIBAgIQT52W2WawmStUwpV8tBV9TTANBgkqhkiG9w0BAQUFADBM MQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkg THRkLjEWMBQGA1UEAxMNVGhhd3RlIFNHQyBDQTAeFw0xMTEwMjYwMDAwMDBaFw0x MzA5MzAyMzU5NTlaMGgxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlh MRYwFAYDVQQHFA1Nb3VudGFpbiBWaWV3MRMwEQYDVQQKFApHb29nbGUgSW5jMRcw FQYDVQQDFA53d3cuZ29vZ2xlLmNvbTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkC gYEA3rcmQ6aZhc04pxUJuc8PycNVjIjujI0oJyRLKl6g2Bb6YRhLz21ggNM1QDJy wI8S2OVOj7my9tkVXlqGMaO6hqpryNlxjMzNJxMenUJdOPanrO/6YvMYgdQkRn8B d3zGKokUmbuYOR2oGfs5AER9G5RqeC1prcB6LPrQ2iASmNMCAwEAAaOB5zCB5DAM BgNVHRMBAf8EAjAAMDYGA1UdHwQvMC0wK6ApoCeGJWh0dHA6Ly9jcmwudGhhd3Rl LmNvbS9UaGF3dGVTR0NDQS5jcmwwKAYDVR0lBCEwHwYIKwYBBQUHAwEGCCsGAQUF BwMCBglghkgBhvhCBAEwcgYIKwYBBQUHAQEEZjBkMCIGCCsGAQUFBzABhhZodHRw Oi8vb2NzcC50aGF3dGUuY29tMD4GCCsGAQUFBzAChjJodHRwOi8vd3d3LnRoYXd0 ZS5jb20vcmVwb3NpdG9yeS9UaGF3dGVfU0dDX0NBLmNydDANBgkqhkiG9w0BAQUF AAOBgQAhrNWuyjSJWsKrUtKyNGadeqvu5nzVfsJcKLt0AMkQH0IT/GmKHiSgAgDp ulvKGQSy068Bsn5fFNum21K5mvMSf3yinDtvmX3qUA12IxL/92ZzKbeVCq3Yi7Le IOkKcGQRCMha8X2e7GmlpdWC1ycenlbN0nbVeSv3JUMcafC4+Q== -----END CERTIFICATE----- subject=/C=US/ST=California/L=Mountain View/O=Google Inc/CN=www.google +.com issuer=/C=ZA/O=Thawte Consulting (Pty) Ltd./CN=Thawte SGC CA --- No client certificate CA names sent --- SSL handshake has read 2130 bytes and written 443 bytes --- New, TLSv1/SSLv3, Cipher is ECDHE-RSA-RC4-SHA Server public key is 1024 bit Secure Renegotiation IS supported Compression: NONE Expansion: NONE SSL-Session: Protocol : TLSv1.2 Cipher : ECDHE-RSA-RC4-SHA Session-ID: 5930A80165EBF4CDA0199A366CB1232C54B4F70B3CEE0690561A95 +14AB8A27EB Session-ID-ctx: Master-Key: A107E655BBC4DC3E28B81CA9986414F2D56E942590F794822EC435 +D3F907C45C7E93D866DF3D082DBE3573278899648D Key-Arg : None PSK identity: None PSK identity hint: None SRP username: None TLS session ticket lifetime hint: 100800 (seconds) TLS session ticket: 0000 - c5 c4 5c ba a7 ff ca 4c-59 f9 5e 08 80 e6 76 3c ..\....LY +.^...v< 0010 - e8 13 92 e8 96 2d 91 fd-e2 ad ff 33 fe ab 16 6d .....-... +..3...m 0020 - 18 15 77 3d f1 d4 b8 24-fe 19 ac 46 b9 69 52 1a ..w=...$. +..F.iR. 0030 - ac db e2 2c 92 33 6c a8-8e 69 f6 3a 65 6d 29 91 ...,.3l.. +i.:em). 0040 - a3 d3 08 6e a7 da 64 f0-88 c7 d4 e3 b4 29 ba 20 ...n..d.. +....). 0050 - a6 31 52 e5 c0 0b 42 b5-da 9c 6d 43 59 17 1e dd .1R...B.. +.mCY... 0060 - 8a 09 0c ee 03 9b 6a a7-87 23 ef d6 2d 61 23 d0 ......j.. +#..-a#. 0070 - 0c 16 c4 69 8c 42 d4 35-00 ae a1 c7 e6 c9 75 2d ...i.B.5. +.....u- 0080 - e2 f7 be 82 93 c2 2c ba-35 67 89 98 c5 8f 47 cb ......,.5 +g....G. 0090 - b4 75 9f c2 .u.. Start Time: 1354196309 Timeout : 300 (sec) Verify return code: 20 (unable to get local issuer certificate) --- read:errno=0

      I don't, yet, understand all of this output, but it appears that the SSL hand shaking dies before there is an attempt to exchange certificates.

      Is the presence or absence of support for "Secure Renegotiation" the key to this problem? If not, what is? And is this something I can have my client adapt to by setting one or another 'ssl_opt' value to something useful.? Or do I have to ask the company that is responsible for the server in question for aide?

      Thanks again

      Ted

        Dude, TL;DR again :) that other guy from that thread i mentioned that forcing SSL3 worked for him ... so I'd try forcing ssl version

        its funny how many devices/software fail at handshaking

        So I've been running into this same problem and it's been driving me nuts. I'm replying to this since my search is turning this node up and maybe this will help other people.

        The actual answer is that your problem is specific to this Verisign certificate, for

        Class 3 Public Primary Certification Authority

        Note that you are in fact getting certificates back from the server; your openssl output has "BEGIN CERTIFICATE" in it after all. What is failing is the certificate chain verification because the Mozilla root certificate bundle that everybody uses has two distinct versions of the same root certificate, i.e., same issuer identity, same public key being advertised, different signature algorithms, MD2 (very old, long-obsolete) vs. SHA1 (still has a few years left, though NIST is getting nervous about this one, too). Verisign is evidently embarrassed by having MD2-signed certificates still out in the wild, would like them to go away, but they can't quite yet, so they've requested that Mozilla distribute both versions of the certificate.

        The problem is that Debian's certificate store — and I'm guessing this applies to everyone who uses openssl to manage certificates — currently can't deal with multiple certificates having the same name, so they're just installing the one with the more secure signature, but the server you're talking to is sending you a chain that ends at the MD2-signed root, which you don't have installed, hence fail.

        You can check if you have this problem (if you're using Debian and it's still May of 2013, then you do) by downloading Mozilla::CA from CPAN, grepping through its certificate file for "Class 3 Public Primary Certification Authority" (without any "Gn" suffix); find both versions — see which one is the MD2-signed one by running them through openssl x509 -text — or (better) run them through

        openssl verify -CApath /etc/ssl/certs
        to see which one isn't properly installed (substitute your own -CApath or -CAfile as necessary; make sure you've got the right one by verifying some of the other certificates).

        My way out of the box is to just install the missing MD2 certificate in your /usr/local/share/ca-certificates and rerun update-ca-certificates. Maybe also do this for "Startcom Certification Authority" which is also duplicated in the same way (they upgraded their sig from SHA1 to SHA256). The problem with this is if Mozilla ever decides to remove these certificates, you won't find out so you'll want to leave a note for yourself for two years from now to check to see if it's still in the Mozilla bundle or if Debian has finally fixed the bug referenced below

        Or you could activate CRL (certificate revocation list) checking (I need to figure out how to do that...)

        Note that this is still a valid root certificate; Mozilla still includes it in its bundle and will probably continue to do so until the last servers using it finally upgrade. They're just not absolutely certain there isn't some way of exploiting the known weaknesses in MD2 (nobody's found one yet) -- my understanding is you still have to break RSA in order to forge a fake certificate chain and the real question is how much longer 1024-bit keys will be viable, and presumably if that ever becomes a problem, there'll be a mass certificate revocation with considerable fanfare... knock wood.

        http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=683403
Re^5: need to fix my installation of IO::Socket::SSL, but how?
by runrig (Abbot) on Nov 29, 2012 at 04:43 UTC
    you can set the LWP verify_hostname option to false, or the environment variable mentioned in Net::HTTPS.

      Thanks. I will look into that, and experiment with it.

      Thanks

      Ted

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1006153]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others imbibing at the Monastery: (6)
As of 2024-03-28 20:13 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found