http://www.perlmonks.org?node_id=1170854

moritz has asked for the wisdom of the Perl Monks concerning the following question:

I just spent several hours debugging some code using LWP::UserAgent. The code could be reduced to this:

use strict; use warnings; use LWP::UserAgent; my $URL = 'https://myhost.local/'; my $ua = LWP::UserAgent->new; $ua->ssl_opts( verify_hostname => 0 ); my $response = $ua->get($URL); if ($response->is_success) { print $response->decoded_content; # or whatever } else { die $response->status_line; }

So pretty much a copy&paste from the LWP::UserAgent docs.

The error message I got was:

Can't connect to myhost.local:443\n\n 500 Can't connect to myhost.local:443 at foo.pl line 15.

After some debugging, strace finally showed me an ENOENT for an SSL CA file, which nudged me into the right direction.

So, my questions are:

This is on a Debian Jessie box with perl 5.20.2, and LWP::UserAgent 6.06.

Replies are listed 'Best First'.
Re: How to get a better error message from LWP::UserAgent on missing SSL certificates?
by hippo (Chancellor) on Aug 31, 2016 at 10:52 UTC
    Why does a missing CA file cause the request to fail, even though I've used ua->ssl_opts( verify_hostname => 0 );?

    Just because you aren't verifying the hostname doesn't mean it won't try to verify the certificate. The CA file is used to verify the certificate which may or may not match the hostname.

    What do I have to do to get a better error message out of LWP::UserAgent?

    Very difficult to get the precise problem as this is handled by the lower-level SSL modules. I'd recommend that you add

    use IO::Socket::SSL 'debug4';

    before you use LWP. That gives loads of output and should at least narrow down the problem to the SSL handshake if nothing more specific. I agree that this is very annoying and would be keen to see better diagnostics passed back up the tree to LWP and thus available to the users. That likely won't be easy.

Re: How to get a better error message from LWP::UserAgent on missing SSL certificates?
by Corion (Pope) on Aug 31, 2016 at 11:48 UTC
Re: How to get a better error message from LWP::UserAgent on missing SSL certificates?
by Krambambuli (Curate) on Aug 31, 2016 at 10:50 UTC
    Just to make sure: was your https server working properly otherwise, could you connect with a browser against the same URL?

        I've tried to reproduce the problem, but it just works here.
        If I hide the CA certificate, I get an error with verify_hostname set, I don't if it is unset or set to 0.
        (Perl 5.22.2, LWP::UserAgent 6.15, LWP::Protocol::https 6.06 on Fedora 24.)
Re: How to get a better error message from LWP::UserAgent on missing SSL certificates?
by Anonymous Monk on Aug 31, 2016 at 09:53 UTC

    Dump the headers. LWP adds extra headers for more diagnostics, they start with Client