Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation

Comment on

( #3333=superdoc: print w/replies, xml ) Need Help??

This is a real puzzle. I am trying to determine why a script works on my workstation (Windows 7), but not on either of two servers running Windows Server 2008. We're using Activestate perl, and the behaviour is identical regardless of whether we use 5.12, 5.14 or 5.16, with the latest release of LWP.

I have a basic script that I used to teach a colleague about Perl web programming. It reproduces the problem exactly. Here it is:

use strict; use Log::Log4perl qw(:easy get_logger); use LWP::UserAgent; use HTTP::Request; use HTTP::Request::Common; use HTTP::Response; use HTTP::Status; $| = 1; my $ua = LWP::UserAgent->new; $ua->add_handler("request_send", sub { log_it("Request: ".shift->dump +); return }); $ua->add_handler("response_done", sub { my $rsp = shift; my @red = $rs +p->redirects; my $r; foreach $r (@red) {log_it("Response redirect: ". +$r)}; log_it("Response last request: ".$rsp->request->uri); log_it("R +esponse headers: ".$rsp->headers_as_string); log_it("Response code: " +.$rsp->code); my $msg = status_message($rsp->code); log_it("Response +status: ".$msg) if defined $msg; log_it("Response status: ".$rsp->sta +tus_line); return }); # Define configuration my $conf = q( log4perl.logger = TRACE, FileApp, ScreenApp log4perl.appender.FileApp = Log::Log4perl::Appender:: +File log4perl.appender.FileApp.filename = lwp.log log4perl.appender.FileApp.layout = PatternLayout log4perl.appender.FileApp.layout.ConversionPattern = %d> %m%n log4perl.appender.ScreenApp = Log::Log4perl::Appender +::Screen log4perl.appender.ScreenApp.stderr = 0 log4perl.appender.ScreenApp.layout = PatternLayout log4perl.appender.ScreenApp.layout.ConversionPattern = %d> %m% +n ); # Initialize logging behaviour Log::Log4perl->init( \$conf ); Log::Log4perl->infiltrate_lwp(); my $logger = get_logger(); my $req_url = $ARGV[0]; if ($req_url =~ m/http/) { $req_url = "$req_url"; } else { $req_url = "http://$req_url"; } # whether you use GET or POST, both shown below, depends on what the # server accepts. try both, preferring POST my $response = $ua->request(POST "$req_url"); my $code = $response->code; if ($code == 405) { $response = $ua->request(GET "$req_url"); } # #my $response = $ua->request(POST "$req_url",/%parms); # print "\n===================================================\n"; # print "The response, as a string, is:\n"; # print $response->as_string; # print "\n===================================================\n"; #if ( $response->is_redirect ) { if (($code > 299) && ($code < 400)) { print $response->previous . " redirected to location " . $respons +e->header('Location') . "\n"; print "The content is: ",$response->content,"\n"; print "\n===================================================\n"; print "The response, as a string, is:\n"; print $response->as_string; print "\n===================================================\n"; $logger->info($response->previous . " redirected to location " . +$response->header('Location')); } else { if ($response->is_success) { print "The content is: ",$response->content,"\n"; $logger->info("The URL $req_url was successfully retrieved."); if (lc($response->content)=~m/'error_response'/){ $logger->warn("Response problematic content: ".$response->conten +t); } } else { $logger->warn("The URL $req_url was not successfully retrieved."); } } sub log_it { my $tmp = shift; $tmp =~ s/\n+/,\t/g; $tmp =~ s/,\t$//; $logger->info($tmp); return; }

Now, if I use the above code, and pass, as the argument either '' or '', All works fine. If, instead, I pass the URL for the querying API I need to use, I get a server error, response code #500. The really bizarre aspect of this is that submitting the same request using MS IE or Firefox works as expected. That suggests that LWP is producing something different from what, or failing to produce something that, the two most common web browsers produce.

The senior tech guy at the company providing the service says that that is happening before the request even gets to his PHP script. He asked that I provide him with all the HTTP headers, along with the complete request, so that he and his team can figure out why only those requests from this script on the two servers fail at his end, and yet the precisely the same code, running on my Windows 7 workstation, succeeds. I can not imagine why the same code would generate different headers depending on which machine it is running on (in all cases, the machines running the script live behind good firewalls).

I guess there are three questions here. First, how do I modify the above script so that instead of actually sending the request to the server in question, it simply writes everything it would normally send to a file instead? This is critical, so that the tech guys at the company providing the API can begin their investigations more thoroughly, with better information. The man was truly baffled as to why a 500 error would be produced by his system. Second, how do I debug this? I did see LWP::Debug, but it's documentation says it is deprecated. I would expect, therefore, that there would be some recent alternative. Is there a directive or variable I can use that would tell LWP to produce debug info, but not actually send the data to the server? And is there a directive to use or a variable that can be set that would result in LWP sending output to standard out representing everything that happens between the client and server, from the point where the two set up an SSL session to the point where the server sends the final response? (I'd really like to be able to do this without manually stepping through the code with a debugger - and I'd like to be able to send such a trace to the folk responsible for the API I am using - knowing at what stage in that dialog the error happens would likely be priceless to them.) Third, does anyone have an thoughts on why precisely the same code would work fine on one machine but fail on another?

It is important to emphasize that this code worked fine on another server since 2007, and only failed once we tried to deploy it to a new machine (in the cloud, if that matters). It is also important to emphasize that making the request seconds after the failed attempt with the script, but using MS IE or Firefox, succeeds; so there is no issue related to connectivity.

Any ideas?



In reply to HTTP error response code 500 using LWP::UserAgent on one site, but not on any other by ted.byers

Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":

  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?

    What's my password?
    Create A New User
    and all is quiet...

    How do I use this? | Other CB clients
    Other Users?
    Others wandering the Monastery: (9)
    As of 2018-06-21 17:12 GMT
    Find Nodes?
      Voting Booth?
      Should cpanminus be part of the standard Perl release?

      Results (118 votes). Check out past polls.