reading from a socket, a little at a time

[Anonymous Monk]

Hi Sagacious Ones

Using IO::Socket::INET in a server context, I've been playing with how to read from a connected client. The client sends binary data of different 'types' (type 1 is one format that is variable in length, type 2 another, but predictable in length). There's an indicator near the beginning of the record that gives an idea of the record type followed by a couple of bytes telling how big the overall length is and a flag.

I thought I could get away with reading the first bytes of the header to determine the type, and then read some more afterward, like so:

   ## Byte 1 - type
   ## Byte 2 - y, flag
   ## Byte 3 - x, 8 + 12 * byte 4
   ## Byte 4 - entry count
   while ( <$client> ) {
        $client->read($raw_header, 6);
        $junk = unpack('H2', $raw_header);
        if ($junk eq "01") {
            print "Variable length is coming!\n";
            $more = unpack('H6', $raw_header);
            print "Three byte header is: $more\n";
            ## Flag byte, noted but not used
            $y = substr( $more, 2, 2 );
            ## Length of data calculated as 8 bytes + 12 * $ec
            $x = substr( $more, 4, 2 );
            print "X, the variable record length is: $x\n";
            print "Y, the flag is: $y\n";
            
            ## Read the entry count byte
            $client->read($raw_header, 2);
            $ec = unpack('H2', $raw_header);
            print "Sub-record Entry Count is: $ec\n";
        }

I was under the impression that I could do a second read() to get the following entry count byte so I could calculate the length and read() the rest of the data correctly... but I know from tcpdump output on the wire that the number $ec gets isn't the one I'm seeing from tcpdump

(A previous version of the code above had me pulling just one more byte and confirmed I was getting the right number for the entry count, but the question about this code segment above stands.)

I was under the impression I could read some bytes, then do something and read some more from where I left off, etc. Looks like I can't, so how would this be properly done?

Thanks,

[Anonyrnous Monk]

A read on a socket does not necessarily read the number of characters requested, but it does return the number of characters actually read (which may be fewer, if not enough data is available at the moment).  Other than that, your impression is correct that a subsequent read would continue where the previous one left off.

BTW, are you sure you want to do a readline (i.e. <$client>) before doing $client->read(...)?

[Anonymous Monk]

I think at this point my mind's gotten to the mushy point and I'm not thinking clearly about what I'm trying to do versus trying to implement. Or rather, I know what I want it to do, but not near getting it implemented right.

[Anonyrnous Monk]

Actually, what I was thinking of in my previous reply is sysread.  read, OTOH, does actually seem to block until the requested number of characters are read, or eof is reached (this is apparently due to the buffering that read involves, as opposed to sysread).  At least that's what a quick test shows.

Here's a simplified demo that (kind of) does what I understood you want to do:

Server:

#!/usr/bin/perl -w
use strict;
use IO::Socket::INET;

my $server = IO::Socket::INET->new(
                 LocalHost => 'localhost',
                 LocalPort => 12345,
                 Proto     => 'tcp',
                 Listen    => SOMAXCONN,
                 Reuse     => 1
             ) or die $!;

print "server started\n";

$SIG{PIPE} = 'IGNORE';

while (1) {
    my $client = $server->accept();
    print "accepted connection\n";
    $client->autoflush(1);
    
    LOOP: for (1..1000) {
        my $count = int(rand 3)+1;
        # 1..3 32-bit BE uints, preceded by the respective count
        my $data = pack "N"x($count+1), $count, ($_)x$count;

        for (split //, $data) {
            last LOOP unless print $client $_;
            select undef, undef, undef, 0.01;  # emulate slow delivery
        }
    }
    close $client;
}

Client:

#!/usr/bin/perl -wl
use strict;
use IO::Socket::INET;

my $sock = IO::Socket::INET->new("localhost:12345") or die $!;

my $buf;
while ($sock->read($buf, 4)) {      # get count
    my $count = unpack "N", $buf;
    $sock->read($buf, $count*4);    # read actual data
    my @data = unpack "N$count", $buf;
    print "$count: @data";
}
close $sock;

Client output:

$ ./888339_c.pl
3: 1 1 1
3: 2 2 2
1: 3
1: 4
1: 5
3: 6 6 6
2: 7 7
3: 8 8 8
2: 9 9
3: 10 10 10
3: 11 11 11
2: 12 12
3: 13 13 13
3: 14 14 14
1: 15
...

[Anonymous Monk]

Hi

Thanks for that, it illustrates the point perfectly! Thanks again!