Re^5: Help with LWP & SSL 500 Internal response error

[afoken]

That 500 sure seems to suggest that 500 Internal Server Error was produced by somebody at some point.

Sure. LWP generates it. This is documented in LWP:

You may also encounter other "Client-XXX" headers. They are all generated by the library internally and are not received from the servers.

and

If the server is not available then the library will generate an internal error response.

The actual code producing such an error is in LWP::UserAgent::_new_response():

sub _new_response {
    my($request, $code, $message, $content) = @_;
    $message ||= HTTP::Status::status_message($code);
    my $response = HTTP::Response->new($code, $message);
    $response->request($request);
    $response->header("Client-Date" => HTTP::Date::time2str(time));
    $response->header("Client-Warning" => "Internal response");
    $response->header("Content-Type" => "text/plain");
    $response->content($content || "$code $message\n");
    return $response;
}

This function is called from only three places, all in LWP::UserAgent:

• at the end of sub simple_request, following an eval BLOCK:

      if ($@) {
      $@ =~ s/ at .* line \d+.*//s;  # remove file/line number
      return _new_response($request, &HTTP::Status::RC_BAD_REQUEST, $@);
      }
  

  Of course, this is the wrong status code.
• in sub send_request, in the unless ($protocol) block, again following an eval BLOCK:

              if ($@) {
                  $@ =~ s/ at .* line \d+.*//s;  # remove file/line numb
  +er
                  $response =  _new_response($request, &HTTP::Status::RC
  +_NOT_IMPLEMENTED, $@);
  

  Again, this is the wrong status code.
• a second time in sub send_request a few lines below, again following an eval BLOCK:

              # we eval, and turn dies into responses below
              eval {
                  $response = $protocol->request($request, $proxy, $arg,
  + $size, $self->{timeout}) ||
              die "No response returned by $protocol";
              };
              if ($@) {
                  if (UNIVERSAL::isa($@, "HTTP::Response")) {
                      $response = $@;
                      $response->request($request);
                  }
                  else {
                      my $full = $@;
                      (my $status = $@) =~ s/\n.*//s;
                      $status =~ s/ at .* line \d+.*//s;  # remove file/
  +line number
                      my $code = ($status =~ s/^(\d\d\d)\s+//) ? $1 : &H
  +TTP::Status::RC_INTERNAL_SERVER_ERROR;
                      $response = _new_response($request, $code, $status
  +, $full);
                  }
              }
  

  This time, it is the right status code. I.e. LWP caught the problem right here, and the protocol handler either returned a status code 500 by itself (unlikely), or no status code at all. In the second case, LWP created a 500 status via _new_response().

The original posting explicitly uses LWP::Protocol::https. Its code (and the used Net::HTTPS) adds a lot of SSL handling, but leaves the actual HTTP handling to LWP::Protocol::http (by inheriting from it). Errors are reporting only via die(), without any attempt to generate a HTTP::Response object.

LWP::Protocol::http contains "read failed" in only two places, both times as die "read failed: $!", one near the code that reads HTTP headers, one near the code that reads the HTTP content.

So noxxi++ seems to be right in Re: Help with LWP & SSL 500 Internal response error, there is a problem inside the SSL connection. The connection is created, but after that, the server seems to stay silent instead of responding. This may be a client-side or a server-side problem. Trying a different HTTPS client as proposed by noxxi is a reasonable way to test for server-side problems.

Alexander