Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

"certificate verify failed" difference between Perl 5.14 and 5.10

by mje (Curate)
on Jun 16, 2011 at 14:33 UTC ( #909968=perlquestion: print w/ replies, xml ) Need Help??
mje has asked for the wisdom of the Perl Monks concerning the following question:

I'm migrating a script from 5.10.0 to 5.14.0 and a GET on a secure web server fails with "certificate verify failed" even though I know the site has a valid certificate:

use LWP::UserAgent; use strict; use warnings; my $ua = LWP::UserAgent->new; my $req = HTTP::Request->new(GET => 'https://www.easysoft.com'); my $res = $ua->request($req); print $res->headers_as_string; print $res->content;

returns content fine in 5.10.0 and headers like this:

Connection: close Date: Thu, 16 Jun 2011 14:22:46 GMT Accept-Ranges: bytes Server: Apache/2.0.54 (Unix) mod_ssl/2.0.54 OpenSSL/0.9.7d mod_perl/1. +999.21 Perl/v5.8.6 Vary: Accept-Encoding Content-Type: text/html; charset=ISO-8859-1 Client-Date: Thu, 16 Jun 2011 14:22:22 GMT Client-Peer: 172.20.100.10:443 Client-Response-Num: 1 Client-SSL-Cert-Issuer: /C=US/O=Equifax/OU=Equifax Secure Certificate +Authority Client-SSL-Cert-Subject: /serialNumber=Paoxfx3blSdh6U20B0CULwa1WF0wpCX +i/C=GB/O=www.easysoft.com/OU=GT68879435/OU=See www.rapidssl.com/resou +rces/cps (c)10/OU=Domain Control Validated - RapidSSL(R)/CN=www.easys +oft.com Client-SSL-Cipher: DHE-RSA-AES256-SHA Client-SSL-Warning: Peer certificate not verified Client-Transfer-Encoding: chunked Content-Style-Type: text/css

Same code in 5.14.0 returns:

Content-Type: text/plain Client-Date: Thu, 16 Jun 2011 14:26:04 GMT Client-Warning: Internal response Can't connect to www.easysoft.com:443

and if I add $ENV{HTTPS_CA_FILE} = "/usr/share/ca-certificates/cacert.org/cacert.org.crt" to the script and run in 5.14.0 I get:

Content-Type: text/plain Client-Date: Thu, 16 Jun 2011 14:26:52 GMT Client-Warning: Internal response Can't connect to www.easysoft.com:443 (certificate verify failed) LWP::Protocol::https::Socket: SSL connect attempt failed with unknown +errorerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certific +ate verify failed at /home/martin/perl5/perlbrew/perls/perl-5.14.0/li +b/site_perl/5.14.0/LWP/Protocol/http.pm line 51.

I had a similar problem connecting to facebook which I was told would be resolved if I installed Mozilla::CA but I already had that installed. In the end I had to I copy the certificates and put them into a "certs" file then a simple "export HTTPS_CA_FILE=/home/martin/certs" made it work. Surely this is not correct.

This is just an example. I'm actually trying to connect to api.betfair.com but this has a valid certificate as well as verified in my browser but api.betfair.com does not return any content so I decided against using it in my example.

Any ideas?

UPDATE Should have mentioned perl 5.10.0 is system Perl on ubuntu and perl 5.14.0 is installed under perlbrew - just in case it makes a difference.

UPDATE2 HTTPS_DEBUG=1 produces output under 5.10.0 and nothing under 5.14.0.

UPDATE3 I had PERL_UNICODE=SAL and unsetting it fixes the problem.

Solution It appears I was missing intermediate certificate 0xeb99629b. Thanks to daxim for putting me on the right track. You can find the details at failed connect or “certificate verify failed” on LWP HTTPS GET

Comment on "certificate verify failed" difference between Perl 5.14 and 5.10
Select or Download Code
Re: "certificate verify failed" difference between Perl 5.14 and 5.10
by Corion (Pope) on Jun 16, 2011 at 14:39 UTC

    You are using LWP::UserAgent with a version above 6.0. You will need to install a certificate authority database like Mozilla::CA (the one by Mozilla).

      As I mentioned in the post, I already have Mozilla::CA:

      perl -MMozilla::CA -le 'print $Mozilla::CA::VERSION' 20110409
Re: "certificate verify failed" difference between Perl 5.14 and 5.10
by Anonymous Monk on Jun 16, 2011 at 16:32 UTC
    You forgot to check the module versions
    $ perl -d:Modlist -S lwp-request -UusSeEd https://www.easysoft.com GET https://www.easysoft.com User-Agent: lwp-request/6.00 libwww-perl/6.02 200 OK Connection: close Date: Thu, 16 Jun 2011 16:31:20 GMT Accept-Ranges: bytes Server: Apache/2.0.54 (Unix) mod_ssl/2.0.54 OpenSSL/0.9.7d mod_perl/1. +999.21 Perl/v5.8.6 Vary: Accept-Encoding Content-Type: text/html; charset=ISO-8859-1 Client-Date: Thu, 16 Jun 2011 16:32:14 GMT Client-Peer: 89.238.155.10:443 Client-Response-Num: 1 Client-SSL-Cert-Issuer: /C=US/O=Equifax/OU=Equifax Secure Certificate +Authority Client-SSL-Cert-Subject: /serialNumber=Paoxfx3blSdh6U20B0CULwa1WF0wpCX +i/C=GB/O=www.easysoft.com/OU=GT68879435/OU=See www.rapidssl.com/resou +rces/cps (c)10/OU=Domain Control Validated - RapidSSL(R)/CN=www.easys +oft.com Client-SSL-Cipher: DHE-RSA-AES256-SHA Client-SSL-Socket-Class: IO::Socket::SSL Client-Transfer-Encoding: chunked Content-Style-Type: text/css Link: </style/style.css>; rel="stylesheet"; type="text/css" Link: </favicon.ico>; rel="icon"; type="image/x-icon" Title: ODBC, JDBC and XML Driver Downloads for Windows, Unix, Linux an +d Mac OS X X-Meta-Description: Easysoft ODBC, JDBC and XML drivers let you access + Oracle, SQL Server, Access, InterBase, Sybase, Firebird, RMS, ISAM, +Coda and Linc from Windows, Unix, Linux, Mac OS X and OpenVMS. X-Meta-Keywords: easysoft,odbc,jdbc,xml,drivers,gateway,bridge,data,da +tabases,files,oracle,interbase,sybase,firebird,microsoft,sql server,a +ccess,coda,openvms,rms,isam,linc X-Meta-Verify-V1: A80XwI68Uh8cIAFxkNSpouY1b8iRx/PeYQcfrBUotks= Use of uninitialized value in split at C:/perl/site/5.12.2/lib/Devel/M +odlist.pm line 49. AutoLoader 5.71 Carp 1.17 Compress::Raw::Zlib 2.035 Config Config_git.pl Config_heavy.pl Cwd 3.33 DynaLoader 1.10 Encode 2.43 Encode::Alias 2.14 Encode::Byte 2.04 Encode::Config 2.05 Encode::Encoding 2.05 Encode::Locale 1.02 Errno 1.11 Exporter 5.64_01 Exporter::Heavy 5.64_01 Fcntl 1.06 File::Basename 2.78 File::Glob 1.07 File::GlobMapper 1.000 File::Spec 3.33 File::Spec::Unix 3.33 File::Spec::Win32 3.33 FileHandle 2.02 Getopt::Long 2.38 HTML::Entities 3.68 HTML::HeadParser 3.66 HTML::Parser 3.68 HTTP::Config 6.00 HTTP::Date 6.00 HTTP::Headers 6.00 HTTP::Message 6.02 HTTP::Request 6.00 HTTP::Response 6.01 HTTP::Status 6.00 IO 1.25_02 IO::Compress::Base::Common 2.035 IO::Compress::Gzip::Constants 2.035 IO::Compress::Zlib::Extra 2.035 IO::File 1.14 IO::Handle 1.28 IO::Seekable 1.1 IO::Socket 1.31 IO::Socket::INET 1.31 IO::Socket::SSL 1.44 IO::Socket::UNIX 1.23 IO::Uncompress::Adapter::Inflate 2.035 IO::Uncompress::Base 2.035 IO::Uncompress::Gunzip 2.035 IO::Uncompress::RawInflate 2.035 LWP 6.02 LWP::MemberMixin LWP::Protocol 6.00 LWP::Protocol::http LWP::Protocol::https LWP::UserAgent 6.02 List::Util 1.23 Mozilla::CA 20110409 Net::HTTP 6.01 Net::HTTP::Methods 6.00 Net::HTTPS 6.00 Net::IDN::Encode 1.1 Net::IDN::Nameprep 1.1 Net::IDN::Punycode 1.000 Net::IDN::Punycode::PP 1.000 Net::SSLeay 1.36 Scalar::Util 1.23 SelectSaver 1.02 Socket 1.87 Storable 2.25 Symbol 1.07 Time::Local 1.2000 URI 1.58 URI::Escape 3.30 URI::Heuristic 4.19 URI::_generic URI::_idna URI::_punycode 0.03 URI::_query URI::_server URI::http URI::https Unicode::Normalize 1.12 Unicode::Stringprep 1.103 Unicode::Stringprep::BiDi 1.10 Unicode::Stringprep::Mapping 1.10 Unicode::Stringprep::Prohibited 1.10 Unicode::Stringprep::Unassigned 1.10 Unicode::Stringprep::_Common 1.10 Win32::API 0.62 Win32::API::Struct 0.62 Win32::API::Type 0.62 XSLoader 0.15 base 2.15 bytes 1.04 constant 1.21 integer 1.00 overload 1.10 unicore::Heavy.pl unicore::To::Fold.pl unicore::To::Lower.pl unicore::lib::Nt::De.pl unicore::lib::Perl::SpacePer.pl utf8 1.08 utf8_heavy.pl vars 1.01 warnings 1.09 warnings::register 1.01
      perl -d:Modlist -S lwp-request -UusSeEd https://www.easysoft.com

      works for me i.e., it downloads the file but my starting example still does not work.

      martin@betdevel:~$ perl -d:Modlist -S lwp-request -UusSeEd https://www +.easysoft .com GET https://www.easysoft.com User-Agent: lwp-request/6.00 libwww-perl/6.02 200 OK Connection: close Date: Thu, 16 Jun 2011 17:05:08 GMT Accept-Ranges: bytes Server: Apache/2.0.54 (Unix) mod_ssl/2.0.54 OpenSSL/0.9.7d mod_perl/1. +999.21 Per l/v5.8.6 Vary: Accept-Encoding Content-Type: text/html; charset=ISO-8859-1 Client-Date: Thu, 16 Jun 2011 17:04:44 GMT Client-Peer: 172.20.100.10:443 Client-Response-Num: 1 Client-SSL-Cert-Issuer: /C=US/O=Equifax/OU=Equifax Secure Certificate +Authority Client-SSL-Cert-Subject: /serialNumber=Paoxfx3blSdh6U20B0CULwa1WF0wpCX +i/C=GB/O=w ww.easysoft.com/OU=GT68879435/OU=See www.rapidssl.com/resources/cps (c +)10/OU=Dom ain Control Validated - RapidSSL(R)/CN=www.easysoft.com Client-SSL-Cipher: DHE-RSA-AES256-SHA Client-SSL-Socket-Class: IO::Socket::SSL Client-Transfer-Encoding: chunked Content-Style-Type: text/css Link: </style/style.css>; rel="stylesheet"; type="text/css" Link: </favicon.ico>; rel="icon"; type="image/x-icon" Title: ODBC, JDBC and XML Driver Downloads for Windows, Unix, Linux an +d Mac OS X X-Meta-Description: Easysoft ODBC, JDBC and XML drivers let you access + Oracle, S QL Server, Access, InterBase, Sybase, Firebird, RMS, ISAM, Coda and Li +nc from Wi ndows, Unix, Linux, Mac OS X and OpenVMS. X-Meta-Keywords: easysoft,odbc,jdbc,xml,drivers,gateway,bridge,data,da +tabases,fi les,oracle,interbase,sybase,firebird,microsoft,sql server,access,coda, +openvms,rm s,isam,linc X-Meta-Verify-V1: A80XwI68Uh8cIAFxkNSpouY1b8iRx/PeYQcfrBUotks= Use of uninitialized value in split at /home/martin/perl5/perlbrew/per +ls/perl-5. 14.0/lib/site_perl/5.14.0/Devel/Modlist.pm line 49. AutoLoader 5.71 Carp 1.20 Compress::Raw::Zlib 2.033 Config Cwd 3.36 Encode 2.42 Encode::Alias 2.13 Encode::Config 2.05 Encode::Encoding 2.05 Encode::Locale 1.02 Errno 1.13 Exporter 5.64_03 Exporter::Heavy 5.64_03 Fcntl 1.11 File::Basename 2.82 File::Glob 1.12 File::GlobMapper 1.000 File::Spec 3.33 File::Spec::Unix 3.33 Getopt::Long 2.38 HTML::Entities 3.68 HTML::HeadParser 3.66 HTML::Parser 3.68 HTTP::Config 6.00 HTTP::Date 6.00 HTTP::Headers 6.00 HTTP::Message 6.02 HTTP::Request 6.00 HTTP::Response 6.01 HTTP::Status 6.00 I18N::Langinfo 0.08 IO 1.25_04 IO::Compress::Base::Common 2.033 IO::Compress::Gzip::Constants 2.033 IO::Compress::Zlib::Extra 2.033 IO::File 1.15 IO::Handle 1.31 IO::Seekable 1.1 IO::Socket 1.32 IO::Socket::INET 1.31 IO::Socket::SSL 1.44 IO::Socket::UNIX 1.23 IO::Uncompress::Adapter::Inflate 2.033 IO::Uncompress::Base 2.033 IO::Uncompress::Gunzip 2.033 IO::Uncompress::RawInflate 2.033 LWP 6.02 LWP::MemberMixin LWP::Protocol 6.00 LWP::Protocol::http LWP::Protocol::https 6.02 LWP::UserAgent 6.02 List::Util 1.23 Mozilla::CA 20110409 Net::HTTP 6.01 Net::HTTP::Methods 6.00 Net::HTTPS 6.00 Net::SSLeay 1.36 Scalar::Util 1.23 SelectSaver 1.02 Socket 1.94 Storable 2.27 Symbol 1.07 Time::Local 1.2000 URI 1.58 URI::Escape 3.30 URI::Heuristic 4.19 URI::_generic URI::_idna URI::_punycode 0.03 URI::_query URI::_server URI::http URI::https XSLoader 0.13 base 2.16 bytes 1.04 constant 1.21 feature 1.20 integer 1.00 overload 1.13 unicore::Heavy.pl unicore::To::Lower.pl unicore::lib::Nt::De.pl unicore::lib::Perl::SpacePer.pl unicore::lib::Perl::Word.pl utf8 1.09 utf8_heavy.pl vars 1.02 warnings 1.12 warnings::register 1.02
        works for me i.e., it downloads the file but my starting example still does not work.

        Whoa, that doesn't make sense.

        lwp-download is essentially identical to your program (except the -d option says don't print the content).

        I have OpenSSL 1.0.0d

        Both work for me :/

Re: "certificate verify failed" difference between Perl 5.14 and 5.10
by ikegami (Pope) on Jun 16, 2011 at 16:34 UTC

    Works for me.

    LWP 6.02, 5.14.0, ActiveState, x86, Windows

    LWP 6.02, 5.12.1, perlbrew, x86_64, Linux, OpenSSL-0.9.8g

    The certificate is by Equifax, and Equifax is in the .pem.

    Check that Mozilla::CA::SSL_ca_file() returns the right file and that you have permission to read the file.

      $ perl -MMozilla::CA -le 'print Mozilla::CA::SSL_ca_file();' /home/martin/perl5/perlbrew/perls/perl-5.14.0/lib/site_perl/5.14.0/Moz +illa/CA/ca cert.pem $ ls -la /home/martin/perl5/perlbrew/perls/perl-5.14.0/lib/site_perl/5 +.14.0/Mozilla/CA/cacert.pem -r--r--r-- 1 martin root 256791 2011-04-09 16:23 /home/martin/perl5/pe +rlbrew/perls/perl-5.14.0/lib/site_perl/5.14.0/Mozilla/CA/cacert.pem $ perl -MLWP -le 'print $LWP::VERSION;' 6.02

      and I can cat the pem file as the user I am running the script as.

        It could be a problem in the OpenSSL library??? Or maybe the .pem is corrupted??? (Line endings???)
Re: "certificate verify failed" difference between Perl 5.14 and 5.10
by ikegami (Pope) on Jun 16, 2011 at 16:37 UTC

    Works for me.

    LWP 6.02, 5.14.0, ActiveState, x86, Windows

    LWP 6.02, 5.12.1, perlbrew, x86_64, Linux, OpenSSL-0.9.8g

    The certificate is by Equifax, and Equifax is in the .pem.

    Check that Mozilla::CA::SSL_ca_file() returns the right file and that you have permission to read the file.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://909968]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others having an uproarious good time at the Monastery: (6)
As of 2014-12-27 10:54 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (177 votes), past polls