Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister

Re^6: multithreaded tcp listener with IO::Socket

by Random_Walk (Prior)
on May 18, 2006 at 12:59 UTC ( #550233=note: print w/replies, xml ) Need Help??

in reply to Re^5: multithreaded tcp listener with IO::Socket
in thread multithreaded tcp listener with IO::Socket

Hi BrowserUk,

I am still having a little trouble with my socket handles. I put the fileno an a queue and the treah picks it up OK. The problem comes down to this line in the handler thread
open my $socket, '<&=', $fileno;

The socket of course is opened read only but I need to write an 'OK' back to it when I have got the event. I tried
open my $socket, '+<&=', $fileno;

still no luck. Any idea how I can re-open it read/write ?

Thanks a million for your help, there is a $favourite_drink waiting for you any time you are in Amsterdam,

Pereant, qui ante nos nostra dixerunt!

Replies are listed 'Best First'.
Re^7: multithreaded tcp listener with IO::Socket
by BrowserUk (Pope) on May 18, 2006 at 13:31 UTC

    You need a dot ('.'), not a comma (',') (and don't forget some error handling :).

    open my $socket, '+<&=' . $fileno or die $!; ........................^

    For an explaination of the syntax and what it is doing, see perlopentut and the section entitled "Re-Opening Files (dups)".

    Sorry for not posting the code I promised. I've been trying to work it into a proper module. It works, but need substantial extra testing and documentation. I'll post it below as it is in it's current state.

    The usage will be something like this:

    my $server = threads::Server->new( LocalPort => 9000, Pool => 5, Accept => sub { printf "Accepted connection from %s:%d on port:%d\n", $_->peerhost, $_->peerport, $_->sockport; return; }, Thread => sub { my( $client, $Qout, @args ) = @_; while( <$client> ) { chomp; ##1 warnf "Got '%s'\n", $_; $Qout->enqueue( $_ ); print $client 'Ack'; } }, Common => sub { my $Q = shift; while( $Q->dequeue() ) { #1 warnf "Processing '$_'"; } }, ); $server->Run;

    That needs explaination, documentation, a lot of work, and relies on another of my own modules (a ripoff of theDamian's Smart::Comments), but it might provide some ideas for you.

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

      hello, first of all thanks a lot for posting that code, it has been of great help for me.

      Now, I found I need to make some changes for it to work. I'm starting a process to serve http request on a port. Client is doing basically this:

      $ua = LWP::UserAgent->new; my $req = HTTP::Request->new(GET => "http://localhost:8998/foo"); $req->authorization_basic('fo', 'fi'); my $res = $ua->request($req) ;

      The thing is it looks I really need to close the socket in sub Run after the sub _thread opened the dequeued file_no. . In other words, instead of :

      while( $Qclean->pending ) { my $fno = $Qclean->dequeue(); close delete $self->{ Clients }{ $fno }; }

      I need to do:

      my $fno = $Qclean->dequeue(); close delete $self->{ Clients }{ $fno };

      before letting it arrive to the "local $_ = $server->accept " again.

      If I don't do that,then looks like the "close" of the fd I get in sub _thread does not really closes the socket? The thing is "$ua->request($req)" at the client does not return until I kill the server process if I don't actually close the socket in sub Run.

        This is untested as I have a bunch of guys with chainsaws and an industrial-sized wood-chipper making enough noise just outside my window to cause the dead to wake and migrate.

        Try substituting these replacements:

        sub Run { my $self = $_[0]; my( $server, $Qwork, $Qclean, $accept ) = @{ $self }{ qw[ Socket Qwork Qclean Accept ] }; while( local $_ = $server->accept ) { my $fileno = fileno $_; $self->{ Clients }{ $fileno } = $_; my $clientArgs = eval{ join chr(0), $accept->() } || ''; $Qwork->enqueue( "$fileno\0$clientArgs" ); while( my $fno = $Qclean->dequeue() ) { #warnf "Cleanup of %d\n", $fno; close delete $self->{ Clients }{ $fno }; } } } sub _thread { #warnf "Starting thread %d\n", threads->tid; my( $userCode, $Qwork, $Qin, $Qclean ) = @_; while( my $work = $Qwork->dequeue() ) { my( $fileno, @args ) = split chr(0), $work; open my $client, '+<&=' . $fileno or cluck "Failed to dup $fileno in ${ \threads->tid } : $! +\n" and next; $Qclean->enqueue( $fileno ); $userCode->( $client, $Qin, @args ); close $client; } }

        The basic problem is that your client won't attempt to move on to making the next connection until both copies of the socket are closed at the server; but the Accept loop copy of the socket won't get closed until someone (your client or another client), make another connection. In a reasonably active system, another client connecting causes the accept loop to cycle and the cleanup occurs; but on a system with only a single client that obviously can't happen.

        The solution above avoids that by having the accept loop wait for the client thread to queue the fileno back for cleanup, and then closing its copy of the socket immediately. I've also made the client thread queue the fileno back as soon as it has duped it, to minimise the impact upon the accept loop. It's not a perfect solution for performance, but then using Perl for a server is never going to be the ultimate high performance solution.

        I'll try to test this locally, and maybe come up with something better, once the ambient nose level here drops below 100dB; but please do feed back your findings.

        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://550233]
[Jeri]: heyyyy!!!!
[Jeri]: I'm about to leave work!!!
[Jeri]: wooooo hooooo!

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (5)
As of 2017-02-23 21:49 GMT
Find Nodes?
    Voting Booth?
    Before electricity was invented, what was the Electric Eel called?

    Results (351 votes). Check out past polls.