This is a real puzzle. I am trying to determine why a script works on my workstation (Windows 7), but not on either of two servers running Windows Server 2008. We're using Activestate perl, and the behaviour is identical regardless of whether we use 5.12, 5.14 or 5.16, with the latest release of LWP.

I have a basic script that I used to teach a colleague about Perl web programming. It reproduces the problem exactly. Here it is:

use strict;
use Log::Log4perl qw(:easy get_logger);
use LWP::UserAgent;
use HTTP::Request;
use HTTP::Request::Common;
use HTTP::Response;
use HTTP::Status;

$| = 1;

my $ua = LWP::UserAgent->new;

$ua->add_handler("request_send",  sub { log_it("Request: ".shift->dump
+); return });
$ua->add_handler("response_done", sub { my $rsp = shift; my @red = $rs
+p->redirects; my $r; foreach $r (@red) {log_it("Response redirect: ".
+$r)}; log_it("Response last request: ".$rsp->request->uri); log_it("R
+esponse headers: ".$rsp->headers_as_string); log_it("Response code: "
+.$rsp->code); my $msg = status_message($rsp->code); log_it("Response 
+status: ".$msg) if defined $msg; log_it("Response status: ".$rsp->sta
+tus_line); return });

# Define configuration
my $conf = q(
        log4perl.logger                    = TRACE, FileApp, ScreenApp
        log4perl.appender.FileApp          = Log::Log4perl::Appender::
+File
        log4perl.appender.FileApp.filename = lwp.log
        log4perl.appender.FileApp.layout   = PatternLayout
        log4perl.appender.FileApp.layout.ConversionPattern = %d> %m%n
        log4perl.appender.ScreenApp          = Log::Log4perl::Appender
+::Screen
        log4perl.appender.ScreenApp.stderr   = 0
        log4perl.appender.ScreenApp.layout   = PatternLayout
        log4perl.appender.ScreenApp.layout.ConversionPattern = %d> %m%
+n
);

# Initialize logging behaviour
Log::Log4perl->init( \$conf );
Log::Log4perl->infiltrate_lwp();

my $logger = get_logger();

my $req_url = $ARGV[0];
if ($req_url =~ m/http/) {
  $req_url = "$req_url";
} else {
  $req_url = "http://$req_url";
}


# whether you use GET or POST, both shown below, depends on what the
# server accepts.  try both, preferring POST
my $response = $ua->request(POST "$req_url");
my $code = $response->code;
if ($code == 405) {
  $response = $ua->request(GET "$req_url");
}
#
#my $response = $ua->request(POST "$req_url",/%parms);
#    print "\n===================================================\n";
#    print "The response, as a string, is:\n";
#    print $response->as_string;
#    print "\n===================================================\n";

#if ( $response->is_redirect  ) {
if (($code > 299) && ($code < 400)) {
    print $response->previous . " redirected to location " .  $respons
+e->header('Location') .  "\n";
    print "The content is: ",$response->content,"\n";
    print "\n===================================================\n";
    print "The response, as a string, is:\n";
    print $response->as_string;
    print "\n===================================================\n";
    $logger->info($response->previous . " redirected to location " .  
+$response->header('Location'));
} else {
  if ($response->is_success) {
    print "The content is: ",$response->content,"\n";
    $logger->info("The URL $req_url was successfully retrieved.");
    if (lc($response->content)=~m/'error_response'/){
      $logger->warn("Response problematic content: ".$response->conten
+t);
    }
  } else {
    $logger->warn("The URL $req_url was not successfully retrieved.");
  }
}


sub log_it {
  my $tmp = shift;
  $tmp =~ s/\n+/,\t/g;
  $tmp =~ s/,\t$//;
  $logger->info($tmp);
  return;
}
[download]

Now, if I use the above code, and pass, as the argument either 'www.google.ca' or 'https://www.google.ca', All works fine. If, instead, I pass the URL for the querying API I need to use, I get a server error, response code #500. The really bizarre aspect of this is that submitting the same request using MS IE or Firefox works as expected. That suggests that LWP is producing something different from what, or failing to produce something that, the two most common web browsers produce.

The senior tech guy at the company providing the service says that that is happening before the request even gets to his PHP script. He asked that I provide him with all the HTTP headers, along with the complete request, so that he and his team can figure out why only those requests from this script on the two servers fail at his end, and yet the precisely the same code, running on my Windows 7 workstation, succeeds. I can not imagine why the same code would generate different headers depending on which machine it is running on (in all cases, the machines running the script live behind good firewalls).

I guess there are three questions here. First, how do I modify the above script so that instead of actually sending the request to the server in question, it simply writes everything it would normally send to a file instead? This is critical, so that the tech guys at the company providing the API can begin their investigations more thoroughly, with better information. The man was truly baffled as to why a 500 error would be produced by his system. Second, how do I debug this? I did see LWP::Debug, but it's documentation says it is deprecated. I would expect, therefore, that there would be some recent alternative. Is there a directive or variable I can use that would tell LWP to produce debug info, but not actually send the data to the server? And is there a directive to use or a variable that can be set that would result in LWP sending output to standard out representing everything that happens between the client and server, from the point where the two set up an SSL session to the point where the server sends the final response? (I'd really like to be able to do this without manually stepping through the code with a debugger - and I'd like to be able to send such a trace to the folk responsible for the API I am using - knowing at what stage in that dialog the error happens would likely be priceless to them.) Third, does anyone have an thoughts on why precisely the same code would work fine on one machine but fail on another?

It is important to emphasize that this code worked fine on another server since 2007, and only failed once we tried to deploy it to a new machine (in the cloud, if that matters). It is also important to emphasize that making the request seconds after the failed attempt with the script, but using MS IE or Firefox, succeeds; so there is no issue related to connectivity.

Any ideas?

Thanks

Ted