Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot

Need slow web client

by McA (Priest)
on Mar 21, 2013 at 09:43 UTC ( #1024694=perlquestion: print w/replies, xml ) Need Help??
McA has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks,

I want to test the buffering capabilities of a reverse proxy. This proxy should be able to pull the whole HTTP response from the application backend and send it to a slow web client freeing the application server process for "real work".

Therefore I need a webclient with handbrakes on: One which reads from socket (configurable) slowly.

Some ideas how to achieve this?

Best regards

Replies are listed 'Best First'.
Re: Need slow web client
by daxim (Chaplain) on Mar 21, 2013 at 09:48 UTC
    1. LWP::UserAgent with :content_cb and put a sleep in there or something.

    2. HTTP::Proxy::BodyFilter::simple and put a sleep in there or something. This is going to be useful for testing other aspects of your proxy, too, as you can tamper with many more aspects of the HTTP request/response pair.

    edit: added LWP

      Thank you. I'll have a look at these.


        Note that HTTP::Proxy doesn't "do" https (yet).

        Enjoy, Have FUN! H.Merijn
Re: Need slow web client
by rjt (Deacon) on Mar 21, 2013 at 10:22 UTC

    Before I get into Perl specifics, I would highly recommend you take a look at Net::Packet::Dump or plain old tcpdump if you're on a compatible system, so you can see what's actually going on between client and server at a packet level. Simply slowing down the client may not have much bearing on the server/proxy, as the TCP/IP stack will buffer sent packets, intermediate routers can shape traffic, and the specific behavior of your proxy (including network settings in the OS/kernel) can have a great impact.

    Particularly if you are serving small requests, the speed of the client may have little to do with freeing up the server process, and main thing you free the server process from in any case is an open socket, which may or may not be significant. If you think you're also freeing up server RAM or CPU cycles by offloading to a proxy, I'd urge you to benchmark this thoroughly before introducing another hop into your architecture, if you haven't already done so.

    You talk about freeing the "server process". If your use of the word "process" implies that the proxy would be another application running on the same (virtual) machine, benchmarking will be even more important.

    I know I'm now way off-topic from a Perl point of view here, but we PerlMonks pride ourselves on being equal opportunity discussors sometimes. :-) The short version is: when trying to squeeze performance from network servers, I've found thorough network and server analysis under load is the only way to go. I'd like to know more about the analysis you've done, as there are many different ways to load test applications, and many definitions of "slow client" (the usual suspects being limited bandwidth, high latency, packet loss, combination of all three).

      Thank you for your comment. But I really want to do a synthetic check whether the proxy is buffering (like promised) the output of the application server (which resides on a different machine). I do agree that this behaviour is only meaningful with returning big responses.

      Personally I like this kind of comment hopefully broaden someone's view.


        But I really want to do a synthetic check whether the proxy is buffering (like promised) the output of the application server (which resides on a different machine).

        Maybe I'm missing something, but can you simply check the web server logs to confirm the request came from the proxy instead of the endpoint? And/or check the proxy logs (perhaps increasing the log level for server connection/caching details)? That should at least answer that question. Most proxies also allow you to customize HTTP headers to some extent, so you can even stick some metadata in there to help get more objective results in your testing.

        For a very simple "slow client", the LWP::UserAgent callback suggestion is reasonable for a starting point.

Re: Need slow web client
by kschwab (Priest) on Mar 21, 2013 at 22:29 UTC

      Hi kschwab,

      my initial question is long ago. Now I had the time to do what I wanted to do. I looked at trickle and it does exactly what I wanted.

      Thank you and a ++ for that hint.

      Best regards

Re: Need slow web client
by bulk88 (Priest) on Mar 22, 2013 at 22:58 UTC

      Good morning,

      I have to agree, but I never said that I want this architecture to protect against SYN flood attacks... ;-)


Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1024694]
Approved by Old_Gray_Bear
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (3)
As of 2017-10-20 00:30 GMT
Find Nodes?
    Voting Booth?
    My fridge is mostly full of:

    Results (258 votes). Check out past polls.