Http Clients errors from LWP::Protocol::http after module upgrade

[tharple]

Performance/load testing a MS Web service, where the data generator client machine has ~100 perl scripts reading data from a local socket and then writing it to the IIS 7.5 web server. The data generator is WIN 2008 R2 with 16 core 32GB server with qty 10 Gigabit adapters. The Web server is WIN 2008 R2 with 8 core 4GB server with qty 1 Gigabit adapters. The messages are sent out via https and are distributed over all adapters. I have successfully uploaded 100,000's of 2 MB messages, but was occasionally receiving the following message:

500 write failed: Content-Type: text/plain client-date: Wed, 17 Oct 20
+12 16:02:37 GMT client-warning: Internal response 500 write failed:

My investigation lead to Stacktrace/Perlmonks suggesting an upgrade of Net::SSLeay and Bundle:LWP After doing so my level of stability has decrease and I now receive significantly more of the following error:

500 Can't use an undefined value as a SCALAR reference
Content-Type: text/plain
client-date: Thu, 18 Oct 2012 18:15:59 GMT
client-warning: Internal response

Can't use an undefined value as a SCALAR reference at C:/strawberry/pe
+rl/site/lib/LWP/Protocol/http.pm line 353.
Results:Can't use an undefined value as a SCALAR reference at C:/straw
+berry/perl/site/lib/LWP/Protocol/http.pm line 353.

I added a debug statement prior to the error line in c:/strawberry/perl/site/lib/LWP/Protocol/http.pm:

        print Dumper $wbits unless $wbits;
        if (defined($wbits) && $wbits =~ /[^\0]/) {

I only get the error when Dumper says $wbits is undef. It looks like $wbits should fail the defined call and not check the match condition.??? The perl scripts are separate processes started via Win32::Process::Create. Here is the code fragment making the calls:

use strict;
use warnings;
use IO::Socket;
use LWP::UserAgent;
use LWP::Protocol::http;
use HTTP::Request;
use Data::Dumper;
use Dis::Time;
use Storable qw(store_fd fd_retrieve);
$ENV{HTTPS_DEBUG} = 1;

....
    my $userAgent = LWP::UserAgent->new(agent => 'ClientSim.pl/0.2', 
                        keep_alive => 1,
                        local_address => $localAddr,
                        ssl_opts => { verify_hostname => 0 },
                        timeout => 1,
        );
    push(@LWP::Protocol::http::EXTRA_SOCK_OPTS, SendTE => 0);
    my $heads = HTTP::Headers->new(
        Content_Type => $data->{Soap}{Content},
        Accept_Encoding => $data->{Soap}{Encoding},
        Expect => '100-continue',
        SOAPAction => $data->{Soap}{Ns}{tem}.$data->{Soap}{Method},
        );
    print "Executing: HTTP::Request\n"  if $debug;
    my $req = HTTP::Request->new("POST", $data->{Soap}{Uri},$heads, $d
+ata->{Message});
    $data->{Retry} = 0;
    while(1) 
    {
        $data->{Start} = Dis::Time->new();
        print "Executing: HTTP::Response\n"  if $debug;
        $data->{Response} = $userAgent->request($req);
        my $code = $data->{Response}->code;
        print "Responce code $code Recieved\n" if $debug;
        last if ("200" eq $code || "500" ne $code || $data->{Retry}>=$
+retry);
        $data->{Retry}++;
        print $data->{Response}->as_string,"Retrying...\n" if $debug;
    }
....

The retry loop and having the timeout set to 1 are workarounds added to get around the issue. This being such a strange error and in an LWP module is my motivation for the posing the question.

perl -V
Summary of my perl5 (revision 5 version 12 subversion 3) configuration
+:

  Platform:
    osname=MSWin32, osvers=6.1, archname=MSWin32-x64-multi-thread
    uname='Win32 strawberryperl 5.12.3.0 #1 Sun May 15 09:43:50 2011 x
+64'
    config_args='undef'
    hint=recommended, useposix=true, d_sigaction=undef
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=und
+ef
    use64bitint=define, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags =' -s -O2 -DWIN32 -DHAVE_DES_FCRYPT -DWIN64 -DCO
+NSERVATIVE  -DUSE_SITECUSTOMIZE -DPERL_IMPLICIT_CONTEXT -DPERL
_IMPLICIT_SYS -fno-strict-aliasing -mms-bitfields -DPERL_MSVCRT_READFI
+X',
    optimize='-s -O2',
    cppflags='-DWIN32'
    ccversion='', gccversion='4.4.3', gccosandvers=''
    intsize=4, longsize=4, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=1
+2
    ivtype='long long', ivsize=8, nvtype='double', nvsize=8, Off_t='lo
+ng long', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='g++', ldflags ='-s -L"C:\strawberry\perl\lib\CORE" -L"C:\straw
+berry\c\lib"'
    libpth=C:\strawberry\c\lib C:\strawberry\c\x86_64-w64-mingw32\lib
    libs=-lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 
+-ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws
2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
    perllibs=-lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdl
+g32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid
-lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
    libc=, so=dll, useshrplib=true, libperl=libperl512.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-mdll -s -L"C:\strawberry\perl\lib\CORE
+" -L"C:\strawberry\c\lib"'


Characteristics of this binary (from libperl):
  Compile-time options: MULTIPLICITY PERL_DONT_CREATE_GVSV
                        PERL_IMPLICIT_CONTEXT PERL_IMPLICIT_SYS
                        PERL_MALLOC_WRAP PL_OP_SLAB_ALLOC USE_64_BIT_I
+NT
                        USE_ITHREADS USE_LARGE_FILES USE_PERLIO USE_PE
+RL_ATOF
                        USE_SITECUSTOMIZE
  Built under MSWin32
  Compiled at May 15 2011 09:52:35
  %ENV:
    PERL_JSON_BACKEND="JSON::XS"
    PERL_YAML_BACKEND="YAML"
  @INC:
    C:/strawberry/perl/site/lib
    C:/strawberry/perl/vendor/lib
    C:/strawberry/perl/lib
    .

And my question are...

defined($wbits) && $wbits =~ /^\0/) {

In what situation can the 1st predicate pass when $wbits is undef?

Or why would && not short circuit correctly?

Is there something obvious that I am missing, that you see?

If this looks like a real issue, how can I assist is further troubleshooting this?

[mbethke]

Hi tharple,

that's a very strange error.

In what situation can the 1st predicate pass when $wbits is undef?
Or why would && not short circuit correctly?

I can't think of any at all, that would make both the defined() test and && pretty pointless. What's more, line line doesn't match the error message. If something made defined() pass an undefined value, I'd have expected "Use of uninitialized value $fbits in pattern match (m//)", however the error speaks of "SCALAR reference" as if some dereferencing was going on there, which there isn't. Perl 5.14+ does some automatic dereferencing in some situations but with 5.12 that shouldn't be the case either. Frankly I don't have a clue. Perhaps something is screwing¹ with your compiled code. I'd try and split up that condition to see what exactly is acting up:

if (defined($wbits)) {
  print "\$wbits is defined: ",Dumper($wbits);
  if($wbits =~ /[^\0]/) {
    print "\$wbits matched\n";
    ...
  } else {
    print "\$wbits did not match\n";
  }
}

¹ To whom it may concern: non-sexual readings preferred here