Beefy Boxes and Bandwidth Generously Provided by pair Networks vroom
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

LWP client, socket, timeout on server side.

by vsespb (Hermit)
on Nov 16, 2012 at 19:41 UTC ( #1004236=perlquestion: print w/ replies, xml ) Need Help??
vsespb has asked for the wisdom of the Perl Monks concerning the following question:

Hello.

I have a multithreaded application, that user 3rd party API (Amazon Glacier actually). It uploads data in several streams, using POST/PUT requests and LWP::UserAgent.

Problem is that if I have many concurrent uploads (which flood a network), I am getting HTTP 408.

1) HTTP 408 comes from server, it's not internal response of LWP::UserAgent - I see server headers, which LWP can't reproduce.

2) If I use computer with higher bandwidth available, I can have more concurrent threads without 408

3) Other users of my software experience that too

4) Other users of this API experience that too. (they use different implementation, I know one Java and one is Python)

Thus I consider that:

a) it's not server side limit for connections per IP.

b) it's server side setup, probably with socket timeout set, global for Amazon Glacier API.

What I wan't to do:

I wan't somehow optimize LWP::UserAgent setup to handle this situation more graceful.

For example I found that it flushes buffers not optimal way:

http://www.nntp.perl.org/group/perl.libwww/2005/12/msg6481.html

Can someone advice something ? What is more important here to avoid 408 server timeouts ?

I can consider using something other, than LWP, even own socket HTTP implementation.

Comment on LWP client, socket, timeout on server side.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1004236]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others meditating upon the Monastery: (7)
As of 2014-04-21 10:13 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (492 votes), past polls