Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

How to get a better error message from LWP::UserAgent on missing SSL certificates?

by moritz (Cardinal)
on Aug 31, 2016 at 09:45 UTC ( #1170854=perlquestion: print w/replies, xml ) Need Help??
moritz has asked for the wisdom of the Perl Monks concerning the following question:

I just spent several hours debugging some code using LWP::UserAgent. The code could be reduced to this:

use strict; use warnings; use LWP::UserAgent; my $URL = 'https://myhost.local/'; my $ua = LWP::UserAgent->new; $ua->ssl_opts( verify_hostname => 0 ); my $response = $ua->get($URL); if ($response->is_success) { print $response->decoded_content; # or whatever } else { die $response->status_line; }

So pretty much a copy&paste from the LWP::UserAgent docs.

The error message I got was:

Can't connect to myhost.local:443\n\n 500 Can't connect to myhost.local:443 at foo.pl line 15.

After some debugging, strace finally showed me an ENOENT for an SSL CA file, which nudged me into the right direction.

So, my questions are:

  • Why does a missing CA file cause the request to fail, even though I've used ua->ssl_opts( verify_hostname => 0 );?
  • What do I have to do to get a better error message out of LWP::UserAgent? Is this a bug in LWP::UserAgent?

This is on a Debian Jessie box with perl 5.20.2, and LWP::UserAgent 6.06.

Replies are listed 'Best First'.
Re: How to get a better error message from LWP::UserAgent on missing SSL certificates?
by hippo (Canon) on Aug 31, 2016 at 10:52 UTC
    Why does a missing CA file cause the request to fail, even though I've used ua->ssl_opts( verify_hostname => 0 );?

    Just because you aren't verifying the hostname doesn't mean it won't try to verify the certificate. The CA file is used to verify the certificate which may or may not match the hostname.

    What do I have to do to get a better error message out of LWP::UserAgent?

    Very difficult to get the precise problem as this is handled by the lower-level SSL modules. I'd recommend that you add

    use IO::Socket::SSL 'debug4';

    before you use LWP. That gives loads of output and should at least narrow down the problem to the SSL handshake if nothing more specific. I agree that this is very annoying and would be keen to see better diagnostics passed back up the tree to LWP and thus available to the users. That likely won't be easy.

Re: How to get a better error message from LWP::UserAgent on missing SSL certificates?
by Corion (Pope) on Aug 31, 2016 at 11:48 UTC
Re: How to get a better error message from LWP::UserAgent on missing SSL certificates?
by Krambambuli (Curate) on Aug 31, 2016 at 10:50 UTC
    Just to make sure: was your https server working properly otherwise, could you connect with a browser against the same URL?

        I've tried to reproduce the problem, but it just works here.
        If I hide the CA certificate, I get an error with verify_hostname set, I don't if it is unset or set to 0.
        (Perl 5.22.2, LWP::UserAgent 6.15, LWP::Protocol::https 6.06 on Fedora 24.)
Re: How to get a better error message from LWP::UserAgent on missing SSL certificates?
by Anonymous Monk on Aug 31, 2016 at 09:53 UTC

    Dump the headers. LWP adds extra headers for more diagnostics, they start with Client

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1170854]
Approved by marto
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (4)
As of 2018-08-16 21:16 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Asked to put a square peg in a round hole, I would:









    Results (172 votes). Check out past polls.

    Notices?