Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

LWP over SSL Hammers Apache

by grantm (Parson)
on Dec 17, 2004 at 01:06 UTC ( [id://415524]=perlmeditation: print w/replies, xml ) Need Help??

We're doing some load testing on a Mason application that runs under SSL on Apache. We have some scripts that use WWW::Mechanize to interact with the application and complete a number of transactions. It all works, but the load on our web server was surprisingly high and the resulting transaction throughput not as high as we'd hoped.

Now I know that SSL is CPU intensive, but I wondered if there was any difference between the load imposed on the server by a request from a browser and a request from a Perl script. To test this, I set up two clients running on my Debian workstation:

  1. Mozilla 1.7.3 using some client-side JavaScript to cause a browser to repeatedly load a small image file (with a random query string added to defeat caching)
  2. A Perl script using WWW::Mechanize to request the same image file in a tight loop.

Now my workstation is not particularly fast and is geographically distant from the web server, so the absolute numbers are not significant, but the relative results are interesting:

Client Protocol Requests/s Server CPU Load
Mozilla HTTP 46.91 0.47%
Mozilla HTTPS 12.24 6.83%
Perl HTTP 28.11 0.65%
Perl HTTPS 5.97 20.50%

The fact that the Perl implementation of HTTP seems to give about half the throughput of Mozilla's highly optimised C implementation doesn't particularly surprise or concern me (and enabling keep_alive might narrow the gap). The fact that the web server has to work three times as hard to respond to half the volume of requests for Perl/HTTPS vs Mozilla/HTTPS is more worrying.

I was vaguely aware that the initial negotiation phase of an SSL interaction was relatively CPU intensive and that to alleviate this, the SSL protocol supports the concept of an SSL session identifier to allow a client and server to reuse the results of an earlier negotiation. In that vein, the POD for Net::SSLeay says:

"this module does not know to issue or serve multiple http requests per connection. This is a serious short coming, but using SSL session cache on your server helps to alleviate the CPU load somewhat"

So I added this to my Apache config:

SSLSessionCache dbm:/var/apache/ssl_session_cache

With this setting in place, the throughput for Mozilla+HTTPS climbed to about 15 requests/s but more importantly, the CPU load dropped to under 1%. Unfortunately, it made no difference at all to the Perl script since the low level Net::SSLeay doesn't do SSL session caching.

The README for IO::Socket::SSL refers to an unofficial later release of Net::SSLeay which does provide the appropriate hooks to allow IO::Socket::SSL to using SSL session caching.

We installed that unofficial release of Net::SSLeay and tested it by looping over this code:

my $client = new IO::Socket::SSL(PeerAddr => "servername", PeerPort +=> "https", SSL_session_cache_size => 100); if (defined $client) { print $client "GET /test/space.gif HTTP/1.0\r\n\r\n"; my @r = <$client>; close $client; } else { warn "SSL socket problem: ", IO::Socket::SSL::errstr(); }

Unfortunately, that still resulted in the same load on the server. Maybe extra parameters are required to enable session caching, I don't know. The next thing I tried was setting up a reusable IO::Socket::SSL::SSL_Context object like this:

my $context = new IO::Socket::SSL::SSL_Context( SSL_version => 'tlsv1', SSL_verify_mode => Net::SSLeay::VERIFY_NONE(), SSL_session_cache_size => 100 );

and reusing it on each request like this:

my $client = new IO::Socket::SSL(PeerAddr => "servername", PeerPort => + "https", SSL_reuse_ctx => $context);

Bingo! the server load dropped down to below 1% and the throughput rate for Perl climbed very slightly to 6.10.

The next question was how we could hack this low-level solution into the multi-layered script (WWW::Mechanise / LWP / IO::Socket::SSL / Net::SSLeay). It turned out to be remarkably easy. In our script, all we had to do was set a global SSL context before using the WWW::Mechanise object:

use IO::Socket::SSL; my $context = new IO::Socket::SSL::SSL_Context( SSL_version => 'tlsv1', SSL_verify_mode => Net::SSLeay::VERIFY_NONE(), SSL_session_cache_size => 100 ); IO::Socket::SSL::set_default_context($context);

Of course whereas the original test script put an unrealistically high load on the server by renegotiating too often; this modified version went too far the other way, by never renegotiating at all. The solution was to discard the default context object after a certain number of requests and create a new one.

Hopefully, these notes will be of use to someone else using LWP with SSL with high request rates.

Replies are listed 'Best First'.
Re: LWP over SSL Hammers Apache
by hakkr (Chaplain) on Dec 17, 2004 at 09:59 UTC
    Hi,

    That is very interesting, real world browser and proxy caching is pretty hard to account for.

    We have been doing similar load testing by using fork and creating system calls direct to lwprequest.

    We found that our test script was limited to creating about 600 parallel requests this way, apparently by the server processing power.

    I was just wondering how many concurrent requests you have managed to generate using this module.

    I am assuming that calling lwprequest directly will allow more parallel requests to be generated than the overhead of using the perl interfaces like this. What do you think is the best way to generate the most parallel requests?
      I was just wondering how many concurrent requests you have managed to generate using this module.

      I'm not sure, but each 'thread' of our test script sleeps between requests to simulate the time taken to fill in each form. We have the load generation scripts running on a number of different client machines.

      I am assuming that calling lwprequest directly will allow more parallel requests to be generated than the overhead of using the perl interfaces like this.

      Um no, lwp-request is just a Perl script that uses the LWP module. It's all Perl.

      What do you think is the best way to generate the most parallel requests?

      We're not really interested in raw 'requests per second' numbers since we want to know how many simultaneous user sessions the servers can handle. What constitutes a 'user session' for your app would be quite different to ours.

      The ApacheBench (ab) tool bundled with Apache might be one way to generate many parallel requests. The mod_perl performance tuning guide has other suggestions.

        Um no, lwp-request is just a Perl script that uses the LWP module. It's all Perl.

        Thanks, I had assumed it was a binary of some sort. Very strange they did't use LWP directly .

Re: LWP over SSL Hammers Apache
by slloyd (Hermit) on Dec 17, 2004 at 19:35 UTC
    Seems to me that this is more of a load issue with LWP than with Perl. I use raw socket calls using perl and have not seen the difference you are experiencing.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlmeditation [id://415524]
Approved by kvale
Front-paged by BrowserUk
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others sharing their wisdom with the Monastery: (8)
As of 2024-03-28 07:58 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found