vsespb has asked for the wisdom of the Perl Monks concerning the following question:
I have a multithreaded application, that user 3rd party API (Amazon Glacier actually). It uploads data in several streams, using POST/PUT requests and LWP::UserAgent.
Problem is that if I have many concurrent uploads (which flood a network), I am getting HTTP 408.
1) HTTP 408 comes from server, it's not internal response of LWP::UserAgent - I see server headers, which LWP can't reproduce.
2) If I use computer with higher bandwidth available, I can have more concurrent threads without 408
3) Other users of my software experience that too
4) Other users of this API experience that too. (they use different implementation, I know one Java and one is Python)
Thus I consider that:
a) it's not server side limit for connections per IP.
b) it's server side setup, probably with socket timeout set, global for Amazon Glacier API.
What I wan't to do:
I wan't somehow optimize LWP::UserAgent setup to handle this situation more graceful.
For example I found that it flushes buffers not optimal way:
http://www.nntp.perl.org/group/perl.libwww/2005/12/msg6481.html
Can someone advice something ? What is more important here to avoid 408 server timeouts ?
I can consider using something other, than LWP, even own socket HTTP implementation.