Beefy Boxes and Bandwidth Generously Provided by pair Networks Cowboy Neal with Hat
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Help finding relevant modules to write a port forwarder from pool

by mje (Deacon)
on Oct 08, 2009 at 15:02 UTC ( #799991=perlquestion: print w/ replies, xml ) Need Help??
mje has asked for the wisdom of the Perl Monks concerning the following question:

Anyone know of any modules on CPAN that I could use to implement the following:

Machine A using inetd or a standalone server listens on port N and knows of a pool of servers on machine B listening on ports p1..pn. For each connection on port N on machine A, the server picks an unused (or the next in sequence if all used) pn on machine B and forwards all data.

The incoming connections on port N and machine A, all connect, send small amount of data, wait for a response and disconnect.

I cannot afford to have this running in a single process because connections may block, i.e. the server processes on machine B may accept data, and take some time processing it before returning an answer.

I'm quite happy writing this myself with the low level socket modules but I thought someone must have done most of this in a CPAN module somewhere - just struggling to find relevant ones.

Thanks

Comment on Help finding relevant modules to write a port forwarder from pool
Re: Help finding relevant modules to write a port forwarder from pool
by merlyn (Sage) on Oct 08, 2009 at 15:06 UTC
    You're looking for a "load balancer". If your protocol is HTTP, you're in luck... see PerlBal.

    -- Randal L. Schwartz, Perl hacker

    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

      Thanks merlyn but unfortunately at this stage in the chain the protocol is not HTTP but just a small string of JSON passed in and a string of JSON passed back.

      HTTP is used further up the chain but it would be a lot of work to set up multiple web servers at this stage.

      The ultimate processes receiving the requests are connected to a DB and have a number of prepared statements already prepared so although these could fork, the reconnection time and prepare times would make it a waste and hence the solution I am looking for.

        Check out POE http://poe.perl.org. If it's your first foray, it will take a while to get used to it (Do a search on PM for poe tutorial and you will come up with a discussion). IMHO it would be a better investment to lean POE and do this using it rather than coding it at a low level. Plus POE is non-blocking.

        AnyEvent is another alternative.

        - Markov

        I once wrote a simple-and-stupid protocol-neutral load balancer in Perl and C, where Perl ran as a daemon selecting the best server, and C ran a TCP-to-TCP forwarder.

        For each incoming TCP connection, the C code asked the Perl code for the best server (using UDP), connected to that server, and forwarded bytes in both directions until both sides had closed the connection. The perl code had a list of available servers for each TCP port, and a little monitor function that regularily checked that the servers were still alive.

        That worked quite well with HTTP, telnet (just to test the code), and a propritary protocol for a native API. So, if your JSON passing protocol opens a new TCP connection for each request, this type of load balancer should work for you. It will probably also work if you stuff a few JSON strings through the connection, but it will not work if you keep the TCP connection open as long as possible.

        Unfortunately, I have no permission to publish the code, but feel free to ask for more details.

        Alexander

        --
        Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://799991]
Approved by toolic
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (9)
As of 2014-04-21 05:57 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (491 votes), past polls