Optimizing TCP packet usage with IO::Socket::INET or similar

[Bloehdian]

Hello Monks,

this is a design/problem approach question, therefor no code up to now:

I want to setup a client-server scenario with IO::Socket::INET (or a similar module) to transfer (textual) data to a remote tcp-port using send() method.

The data to be sent is received continuously line by line from a different TCP-port and it should be sent continuously to the target socket as well.

To make transfer as efficient as possible in terms of overhead/payload ratio it is necessary to stuff as much data into tcp packets as possible, having at least padding as possible.

I.e., if MTU is 1500 and assuming TCP-overhead to be 64 bytes, the payload in each packet should be 1436 bytes for each packet (ideally).

Will my original idea

$socket->send( $data );

with $data comprising exactly 1436 bytes work or am I wrong assuming that each send() will put $data into no more than one tcp-packet?

And: as stated, the data to be transmitted comes in line by line as a contiuous stream. It is expected that each line has significantly less bytes than the MTU, but this is not guaranteed. Accumulating enough data lines to fill a tcp-packet and chopping the last line if it would exceed the maximum payload is not that straightforward.

Therefor:

Is there a way to do this automatically/implicitly (using a Perl module)?

Related:

I am thinking about further optimization of the data transfer in terms of low bandwith usage by deploying Net::EasyTCP with compression. Am I right with the assumption that the receiving application on the remote side has to apply the same compression modules, e.g., Compress::Zlib, to uncompress the data or is this somewhere done by the TCP-client?

Cheers

Bloehdian

[BrowserUk]

To make transfer as efficient as possible in terms of overhead/payload ratio it is necessary to stuff as much data into tcp packets as possible, having at least padding as possible.

This cannot be controlled from Perl; nor should you try!

You should just send your data in whatever units you get it in -- lines, blocks whatever -- and let the tcpip stack do its job and coalesce or break up those packets into whatever size chunks it knows will be best for the prevailing conditions of the circuit between you and the destination.

Anything you do at the Perl (or most other languages; barring you drop down to raw ip -- DONT!) will simply be undone and reformed into whatever the stack decides is right. And that can vary from connection to connection, and even within the life of a single connection, depending upon the capabilities of the nodes; the prevailing network loading, the routing; et al.

The mechanisms are very well tried & tested, and have been optimised over decades; anything you try is unlikely to beat them and far more likely to interact badly with them, and result in less optimal performance.

(Take this from someone who has tried :)

[talexb]

You can probably perform some benchmarks for the different strategies and see how they perform for the various parameters that are important for your application (throughput, latency, memory load, CPU load, whatever else you deem important).

As brother BrowserUK has explained, there's virtually nothing you can do from Perl that will affect what your stack will do in getting data from point A to point B.

Depending on how important some of your parameters are, you may find yourself abandoning Perl and going with an implementation written in C. That may take longer to develop, but you will almost certainly get better performance. As always, Your Mileage May Vary.

[BrowserUk]

Depending on how important some of your parameters are, you may find yourself abandoning Perl and going with an implementation written in C. That may take longer to develop, but you will almost certainly get better performance.

Actually, it's very unlikely that dropping into C will do anything to improve (or even affect) the throughput performance of tcp comms.

[Bloehdian]

o.k., folks, Your replies make my life much easier, because I do not have to code anything regarding tcp packet stuffing. I will leave it up to the geniuses who brought TCP to us.

But: Does somebody know anything about the compression issue?

Is it like that, that when I send the data compressed to the socket using Net::EasyTCP, the application on the remote side is responsible for decompressing (i.e., the client cannot "trigger" this). Might seem to be a silly question, but, honestly, I am not a genuine network programmer. ;-)