Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

Threading Client-Server (IO::Socket)

by bloonix (Scribe)
on May 30, 2006 at 12:18 UTC ( #552477=perlquestion: print w/ replies, xml ) Need Help??
bloonix has asked for the wisdom of the Perl Monks concerning the following question:

Hello people,

I'm searching for a solution to build a client-server
environment. My idea - or my vision :) - is the following:

I have a master process and 1-n defined slave processes.
The master listens on a single port and divert incoming
requests from clients to the slave processes. Now I don't
know the best way to do this. One of my considered
solutions is to deal a new port on that a slave is running.
Maybe each slave runs on a own port. Or maybe a slave could
inherit the new socket ($ns = $socket->accept()).

Does someone have links or snippets with examples that
containes parts of my idea?

Update: The background of my question is that I develope
a performance monitor and 1000s of servers deliver their
statistics like processor, memory and network workload to
a central data server.

Update: Sorry people for my bad description. :(
I try to describe it in example steps:

1. listener.pl runs on port 43600
2. 10 server.pl processes runs on ports 43601-43610
3. incoming client requests on port 43600 to listener.pl
4. listener.pl answer the next free port from a server.pl
5. the client reconnect to a server.pl to deliver his statistics

I want to use this way to devide the workload.

Thanks,
Jonny

Comment on Threading Client-Server (IO::Socket)
Re: Threading Client-Server (IO::Socket)
by Random_Walk (Parson) on May 30, 2006 at 13:25 UTC

    Hi Jonny,

    I have been working on exactly this problem lately and got a massive amount of help from BrowserUk++. Please see multithreaded tcp listener with IO::Socket for the discussion and some excellent code Re^7: multithreaded tcp listener with IO::Socket.

    The main trick is in passing the file handles to a pool of threads that handle the connections and a little bit of trickery to stop them going out of scope too soon.

    For my problem I have an architecture like this ...

    1 main/listener Thread listens for connections passes socket via queue cleans up old sockets Pool of receiver threads check data and send OK or ERROR to client, instruction to close socket back to main thread and puts ref to data on another queue for... Router thread Pulls refs off the queue and decides which of several handler threads will get it (may be one of more). spawns handler if it is not yet running then puts ref on queue for that handler. Handler threads do thing like logging, hearbeat monitoring forwarding exceptions to a console etc etc. Self Heartbeat Thread Send periodic heartbeats through its own mechanism to another monitor console so we know it is still working

    Update

    I am also using this for performance monitoring, stats gathering etc. Not sure how varied your data sources are, we have everything from heartbeats and disk space monitors to robots running synthetic transactions on our live web sites. One big step in making it all work was to define a common format for the event data. We have based ours on IBM's "Common Base Event". There is a paper out there called "Cannonical Situation Data Format: The common base event". It makes the rest of the code much simpler.

    Cheers,
    R.

    Pereant, qui ante nos nostra dixerunt!
      Hi R.,

      do you like to tell me how do you gathering the stats?
      Are there any Perlmoduls you use?

      Cheers,
      Jonny

        Hi Jonny

        We have an eclectic mixture of scripts developed over some time. A lot are shell and Perl4, the more recent stuff is Perl5 but I really mean recent, when I started there less than six months ago it was all Perl4. I am trying to encourage use of modules but given the normal security/paranoia/big.org it is not easy to push anything outside core. It also all has to be cross platform (win/aix/linux mostly, some special stuff for mainframes but that is an accepted exception to X-platform rules).

        The stuff we are mostly using outside of core are XML modules (we use J-meter to run synthetic transactions on web pages and it reports in XML) and various DBI modules.

        from core we use the threading modules, IO::*, Time HiRes, Net::Ping, FindBin, File::Find and a handful more I can not recall without greping our source.

        We are building a large library of our own modules, some very in house and some that I am trying to get on CPAN. Ironicaly management love open source (free as in beer) but when I proposed to put some of our modules on CPAN I hit a sort of sand trap of redirection and passive resistance. No one will say no but no one will approve it either.

        So for the most part now it is writing wrappers for legacy code and working on the sexy new core system.

        Drop me a /msg and we can discuss further withouth going to re^12 and filling up nodes with trivialities for others.

        Cheers,
        R.

        Pereant, qui ante nos nostra dixerunt!
Re: Threading Client-Server (IO::Socket)
by jesuashok (Curate) on May 30, 2006 at 13:32 UTC
    Hi

    from your write-up I understood that the client requests will be diverted to slave process, which will deliver the static process.

    If my understanding is right. you no need to follow the architecture at all. because all the slave process will deliver there performance statics.

    If that is the case you will have the slave process running on particular port. You just make the client to access that port and get all the information which are provided by that server.

    suppose If your master server is added with more intelligence like controlling the traffic and so on, then make the server to run on particular port. whenever the client request is hitting with the master just pass the client file descriptor to the slave.

    Create Socket accept connection if ( accept ) { subroutine to divert to slave process }
    "Keep pouring your ideas"
Re: Threading Client-Server (IO::Socket)
by samizdat (Vicar) on May 30, 2006 at 13:38 UTC
    Or try it the UNIX-y way: A Simple Socket Server Using 'inetd'. I found that to be a lot easier to work with. inetd spawns as many different copies on the same port as you need connections. That's why it's called the "super server". :) With inetd, everything from the specified port is sent to <STDIN> and everything from <STDOUT> is sent back. Remarkably powerful tool, makes socketing almost trivial! :D

    Don Wilde
    "There's more than one level to any answer."
Re: Threading Client-Server (IO::Socket)
by merlyn (Sage) on May 30, 2006 at 15:28 UTC
Re: Threading Client-Server (IO::Socket)
by zentara (Archbishop) on May 30, 2006 at 16:13 UTC
Re: Threading Client-Server (IO::Socket)
by bloonix (Scribe) on May 30, 2006 at 19:02 UTC
    Thank you all for your answers. I will take a look into the threads you posted. They looks very interesting! Greez, Jonny

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://552477]
Approved by tinita
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (12)
As of 2014-09-18 20:09 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (124 votes), past polls