Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical

Slow LWP?

by vaevictus (Pilgrim)
on Apr 16, 2009 at 15:06 UTC ( #758012=perlquestion: print w/ replies, xml ) Need Help??
vaevictus has asked for the wisdom of the Perl Monks concerning the following question:

I've got an issue where a client who is using a modem (yes. really, in 2009, aol too) is uploading a document from an application i've been developing an maintaining. Sometimes, however, the webserver appears be timing out or something after a while. Since i have production and development areas, and no real phone line in my office, I decided to start writing a quick LWP based testing simulation... ... but i don't see any way to slow it down. is a socket and sleep my best option? POCO::HTTP maybe? Any other good hacks?

Comment on Slow LWP?
Re: Slow LWP?
by almut (Canon) on Apr 16, 2009 at 15:46 UTC
    ... but i don't see any way to slow it down

    I once had a similar requirement (not LWP, but X11 connections, though).  For this, I simply sent the data through a bidirectional proxy (implemented with IO::Socket and IO::Select), and inserted delays as appropriate. I.e., the inner loop shoveling the data roughly looked like this

    while($sel->can_read) { my $nbytes = sysread( ... ); my $wait_sec = $nbytes / $bandwidth; select(undef,undef,undef, $wait_sec); # delay syswrite( ... ); }

    So, let's say with 10000 bytes being transferred ($nbytes), and with the simulated $bandwidth being 2000 (bytes/sec), a total delay of 5 sec. would be forced upon the transfer. This could be refined to take the real bandwidth of the connection into account (which was large in my case), but in order to just get a feel for how slow something is, it might be sufficient as it is...  Similarly, you could add roundtrip latencies, or play with transfer block sizes, etc.

Re: Slow LWP?
by ig (Vicar) on Apr 16, 2009 at 19:42 UTC
      Slick little insertion therein. ++
      #!/usr/bin/perl -w use IO::Socket; package IO::Socket::INET; sub sysread { my $self = shift; select(undef,undef,undef,0.1); $self->SUPER::sysread(@_); } sub syswrite { my $self = shift; select(undef,undef,undef,0.1); $self->SUPER::syswrite(@_); } package main; # Your LWP code here... use LWP; use LWP::Debug '+'; print LWP::UserAgent->new->request(HTTP::Request->new(GET => ""))->as_string;
Re: Slow LWP?
by brian_d_foy (Abbot) on Apr 16, 2009 at 22:17 UTC

    Curiously, this is the topic of Flavio Poletti's article in the current issue of The Perl Review, using just the features in LWP::UserAgent.

    brian d foy <>
    Subscribe to The Perl Review
Re: Slow LWP?
by quasimojo777 (Initiate) on Apr 17, 2009 at 00:23 UTC
    I hope this isn't frowned upon in the Monastery, but while I have no crushingly awesome PERL way around this problem, I do possess a good memory. I read this post on slashdot a week or two ago. It's about tricking linux into acting like a slow wide area network for testing. Maybe this well help you out. Apologies in advance if cross linking is frowned upon:

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://758012]
Approved by ikegami
Front-paged by derby
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (9)
As of 2014-10-24 11:35 GMT
Find Nodes?
    Voting Booth?

    For retirement, I am banking on:

    Results (131 votes), past polls