Re^6: Does IO::Select work? Anywhere?

[BrowserUk]

To get away from your obsession on readline,

Observation is not obsession.

I can kill the client's xterm, before I send any newline, and the server stays responsive to new and other connections.

Sure you can if you fork, but that's a different solution. Ie. a forking server not a multiplexing server.

Here is a perfectly serviceable equivalent using threads:

#! perl -sw
use strict;
use threads stack_size => 4096;
use threads::shared;
use IO::Socket;

use constant CRLF => chr( 13 ) . chr( 10 );
$/ = $\ = CRLF;

my $server = IO::Socket::INET->new(
    LocalHost => 'localhost',
    LocalPort => 1025, Listen => SOMAXCONN, Reuse => 1,
) or die "Couldn't create listening socket";

while( 1 ) {
    my $client = $server->accept;
    async {
        my $peerhost = $client->peerhost .':'. $client->peerport;
        while( <$client> ){
            chomp;
            print $client $_;
            print "$peerhost:$_";
        }
    }->detach;
}

close $server;

That will (has) run all day and night without problems.

And here is a 'one-liner' that will launch 1000 clients at it each time:

perl -Mthreads=stack_size,4096 -MIO::Socket 
-E"
async{ 
    my$s=new IO::Socket::INET('localhost:1025') or warn $^E, return; 
    say $s 'Hello'; 
    print scalar <$s>; 
    shutdown $s, 2; 
    close $s; 
}->detach for 1 .. 1e6"

But as with all forking servers, it is ridiculously resource intensive for an echo daemon.

[zentara]

This is the simplest usage of IO::Select I can produce. It too, on Linux anyways, is very solid. I can kill -9 clients, and the server dosn't get hung. What do you have to do to make your symptoms appear using this script? You know I'm sure, that multiplexing with IO::Select is only meant for use with short messages; if you are doing large data transfers, you need to use fork or threads. This thread could be also be improved to use sysread instead of <>, but it generally works fine as is.

#!/usr/bin/perl
use IO::Socket;
use IO::Select;

my @sockets;
my $machine_addr = 'localhost';
$main_sock = new IO::Socket::INET(LocalAddr=>$machine_addr,
      LocalPort=>1200,
      Proto=>'tcp',
      Listen=>3,
            Reuse=>1,
            );

die "Could not connect: $!" unless $main_sock;
print "Starting Server\n";

$readable_handles = new IO::Select();
$readable_handles->add($main_sock);

while (1)
{
  ($new_readable) = IO::Select->select($readable_handles, undef, undef
+, 0);

  foreach $sock (@$new_readable)
  {
    if ($sock == $main_sock)
    {
      $new_sock = $sock->accept();
      $readable_handles->add($new_sock);
    }
    else
    {
      $buf = <$sock>;
      if ($buf)
      { 
        print "$buf\n";
        my @sockets = $readable_handles->can_write();
    #print $sock "You sent $buf\n";
    foreach my $sck(@sockets){print $sck "$buf\n";}
        
      }
      else
      {
        $readable_handles->remove($sock);
        close($sock);
      }
    }
  }

}


print "Terminating Server\n";
close $main_sock;
getc();

[BrowserUk]

What do you have to do to make your symptoms appear using this script?

Okay. I'll play :)

1. Start your server in one session.
2. Run this in a second session:

   perl -MIO::Socket -E ' $s=IO::Socket::INET->new("localhost:1200"); $s-
   +>send( "a" ); sleep 1e6 '
   

3. Now start a third session and telnet into your server.

   You'll get a connection and be able to type stuff; but you'll get no replies and none of your telnet input will be displayed on your server's console.

4. Start as many more clients -- telnet or otherwise -- as you like.

   Your server will never see anything from any of them until you kill that perl one-liner. (Or you wait the 11.57 hours DAYS for the sleep to time out.)

The reason is that the send('a') caused select to return a readable file handle. Your server then attempted a readline from that client; but the client never sends a newline, so the readline never returns and your server is dead in the water. The tcpip stack will service connections, but your server will never loop back to accept them.

One bad client and your server is DOS'd. This is the exact scenario that I described above with that "teaching material"; the source of my "obsession with readline".

You know I'm sure, that multiplexing with IO::Select is only meant for use with short messages; if you are doing large data transfers, you need to use fork or threads.

Absolutely not!

If you use recv (or sysread), thus avoiding line-based and buffered IO, you can service huge data packets and small ones; you simply accumulate partial packets in buffers and only process input once you've accumulated enough to satisfy the comms protocol requirements. Whether that is newline (or other character sequence) terminated records or length pre-fixed; or any other mechanism.

(Oh. And don't forget to set the sockets non-blocking!)

[zentara]

Ok, I will put my mind to it. I followed your instructions and yes, it does as you say block. My first thought is isn't it clever the way the MIComplex designed our socket connections, so that they can jam any socket based system they want? ;-)

Even though you denigrated the earlier good professor's code of sysreading 1 byte at a time, in Re^4: Does IO::Select work? Anywhere?

while ($c ne "\n" && ! $endoffile) {
    if (sysread($filehandle, $c, 1) > 0) {
        $retstr = $retstr . $c;
    }
    else {
        $endoffile=1;
    }
    }

maybe therin lies your answer?

The intuitive side of my brain points you to IPC3 buffer limit problem, where you can use syscntl statements. Merlyn says it's effective, except for one case. :-) So I figure, what are the odds? The odds that the last buffer is exactly 4k? 1 in 4k ? Pretty good odds. :-) But then again, I don't fly spaceships either. :)

P.S. I tried your hanging script on the server in Glib based forking server with root messaging and the server handled it just fine. It continued reading messages from clients, and accepted new clients. Sometimes, you just have to learn to use modules instead of reinventing the conveyor belt.