Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

UDP connection

by falstad (Novice)
on May 03, 2012 at 06:28 UTC ( #968647=perlquestion: print w/ replies, xml ) Need Help??
falstad has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks,

I'm new to networking and threading in Perl, and I got the following task which I can't solve.

We have a server which listens on 4 ports. And once it receives a "heartbeat" (a byte containing "01") it sends a response to the IP and port the hb came from (byte containing "25"). This sets up the connection. We have to repeat this in every 10 seconds to keep it alive. After the the server got the first hb it starts to send hexdumps to the client which we have to log into files.

If everything went fine we should have 4 threads signaling the server. 4 other threads listening to the responses and logging the hexdumps.

The problem is that I can't manage to simultaneously listen to the same port and send on it. I have to use UDP and can't afford to miss anything the server sends. Also the code of the server can't be changed.

Update:

I got it to work. Thank you for all the replies. Now, as a finishing touch I have to log the result of the recv to a file as the following pattern:

0A 16 68 00 96 44 01 08 08 05 12 20 3B 00 00 00

Do you have a good idea how can I do the conversion?

Comment on UDP connection
Re: UDP connection
by BrowserUk (Pope) on May 03, 2012 at 06:53 UTC

    Let me see if I got this right.

    • You have a server that requires to receive a heartbeat from it clients every 10 seconds; even while it is concurrently sending data to that client?
    • And it requires to receive that heartbeat via the same port on which it is sending the data?
    • And it does this via separate ports.
    • And you want to write a single client to talk concurrently in both directions to all four ports ?

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    The start of some sanity?

      Thanks for the fast reply.

      Yes, the server requires to receive the heartbeats in every 10 seconds.

      Let me write an example. It's based on a wireshark capture.

      1. The server listens on 4 ports 8020, 8019, 8008, 8003. These ports are reserved (this is a switch online all the time, it doesn't really matter I think).
      2. The client sends the first heartbeat from ports: 53036, 53037, 53038, 53039 to ports 8020, 8019, 8008, 8003. This repeats in every 10 seconds.
      3. When server receives them replies from ports 8020, 8019, 8008, 8003 to ports 53036, 53037, 53038, 53039.
      4. If there are test results ready on the server it sends them to ports 53036, 53037, 53038, 53039.

      If possible I would like / have to write a single client.

      /li

        Hm. It is still not clear from your description that the heartbeats and data transfers take place concurrently.

        1. Does a test result data transfer take longer than 10 seconds?
        2. Do you actually see a heartbeat get sent from client to server, whilst the a data transfer is in progress from server to client?

          (I'm not sure that this is even possible!)

        3. If so, what happens about the servers reply to the heartbeat?

          Does it suspend the data transfer in order to send the reply?

          Or does the reply get delayed until after the transfer completes?

          Or does the server ignore heartbeats sent whilst a data transfer is in progress?

        I suspect that the logic is more likely, the client sends a heartbeat every 10 seconds unless a data transfer is currently in progress.

        If that is the case, the requirements are actually quite simple. But as you've described so far, I don't think they are possible, using threads processes or a select loop.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

        The start of some sanity?

Re: UDP connection
by Marshall (Prior) on May 03, 2012 at 06:55 UTC
    I think there are some design problems here. First, why would you use the most complicated model, threads instead of separate processes for each port?

    This requirement: I have to use UDP and can't afford to miss anything is incompatible with UDP. UDP is a "connectionless", non-error checked protocol. Sure there is going to be data loss - this is inevitable when transferring lots of data. One of my colleagues measured this on one network at 10% packet loss. With UDP there is no "end to end" error checking or retransmission - this can be like shouting into a crowded sports arena.

    Update: I see that BrowserUk hit the send button on a better response than mine. His questions are excellent. Independent of these questions, with UDP some amount of data loss is going to happen and this fundamental requirement of "can't miss anything" cannot be satisfied no matter what when using UDP as the base protocol. Just cannot happen.

      I asked the same thing when I got this task... First of all this will run on an internal network of a big company. They have a reliable network with close to zero packet loss even with UDP. (or something like that xD I'm just a pre-graduate trainee here)

      Legacy software... I have to make a client to match the server.. *sigh*

        Perhaps you were given this task because you are a pre-graduate trainee? It could be a test or hazing ritual.

Re: UDP connection
by Neighbour (Friar) on May 03, 2012 at 08:41 UTC
    Perhaps it helps if you'd start with just 1 'connection'. For example, let's assume the server only listens on port 8003. Recently (AnyEvent::Handle::UDP -- can't get it to receive?) there was a question involving threaded UDP traffic, you might be able to use that.
    Also, it might be a good idea to create a big state-diagram, keeping track of who (server/client) is doing what (waiting for hb, waiting for hb response, waiting for data (hexdumps), sending data) but also when responses will be ignored (as pointed out in earlier comments).
    Next to that, consider what happens if you *do* (and, given that you're told to use UDP, you must expect that at some point you will) lose a packet (at any point in the state-diagram). Which data-loss will result from that, and will your server/client be able to recover from it, or will they get stuck in a state where they're waiting for something which never arrives and need to be restarted completely?
Re: UDP connection
by JavaFan (Canon) on May 03, 2012 at 09:30 UTC
    Unlike some others in this thread, I don't think it's impossible to use UDP to send data in two directions: a heartbeat from the client to the server, and data from the server to the client. In fact, I don't see any particular difficulties -- with IP over 30 years old, we have learned to distinguish between source and destination addresses in IP packets (not that this ever was a problem).

    You want to do something like the following (untested pseudo code):

    ... create sockets ... my @fileno = (fileno(SOCKET1), fileno(SOCKET2), fileno(SOCKET3), fileno(SOCKET4)); my ($rbits, $ebits); vec($rbits, $_, 1) = 1 for @fileno; vec($ebits, $_, 1) = 1 for @fileno; while (1) { my $time_to_next_heartbeat = ...; # How long till we have to send + a hb if ($time_to_next_heartbeat <= 0) { ... send heartbeats ... redo; } if (select (my $rbits = $rbits, undef, my $ebits = $ebits, $time_t +o_next_heartbeat)) { if ($ebits) { ... deal with errors ... } if ($rbits) { foreach my $fileno (@fileno) { if (vec($rbits, $filen0, 1)) { ... read data ... } } } } }
    There's a much simpler solution though, and that may work for you:
    • Start 4 processes. Each process sends a heart beat every 10 seconds (one process for each of the ports).
    • Start another 4 processes. Each of them reads (and logs) whatever it gets from a particular socket.
      untested pseudo code

      If a read is in progress when the 10 seconds times out, then the heartbeat will be delayed until the read completes.

      much simpler solution though

      Two processes will be able to send to the same remote port 'simultaneously' -- thought the tcpip stack will serialise those communications.

      But can two concurrent processes talk from the same local port concurrently?

      Neither scenario seems to fit the OPs exact description does it!


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

      The start of some sanity?

        If a read is in progress when the 10 seconds times out, then the heartbeat will be delayed until the read completes.
        Sure, and if that's a problem, send it after 9.8 seconds, or some other interval. Read in small chuncks if that's required. Or just use a separate process.
        But can two concurrent processes talk from the same local port concurrently?
        Whether or not that's possible is irrelevant (however, this is UDP we're talking about, it's isn't about armies of ogres having to exit a tower through a narrow gate), as that is not what the OP seems to be doing. To quote him:
        The client sends the first heartbeat from ports: 53036, 53037, 53038, 53039 to ports 8020, 8019, 8008, 8003. This repeats in every 10 seconds.
        Oh, and do note that in my sketch, there's just one program anyway.
Re: UDP connection
by sundialsvc4 (Abbot) on May 03, 2012 at 13:31 UTC

    One thread could also simply be dedicated to sending heartbeats every 10 seconds.   If you think that this might interfere with data transfer, record the timestamp each time you send a message to any port, and have the heartbeat process read it.   I would have the heartbeat timer internally set for 3 seconds in this case so that it could always be sure to send a message to any of the ports while not allowing the 10-second window to expire for any.   But since you’re not talking about hundreds of ports here, almost any similar design will do.

Re: UDP connection
by mbethke (Hermit) on May 04, 2012 at 05:15 UTC

    I wish there was less dissing each other and more coding in this thread, but that seems to be how it is when it comes to threads here ...

    As usual I think threading isn't the best approach here as it is prone to subtle errors. That's not to say the following doesn't contain any subtle errors either but that's more likely because I didn't fully understand the specification. Other than that it seems to work fine. If you're unfamiliar with POE: it's an event based framework; each of the hash keys in the inline_states parameter to POE::Session->create is a named event handler. _start gets called first and opens four sockets and at the same time fires off a heartbeat event. The heartbeat handler first starts a new timer to be called again after 10 seconds with the same server port as an argument, then it sends the "\x01" message. Whenever a socket receives something, input is called and receives an array of inputs (usually only one) in $_ARG0; you'd have to flesh this one out of course. Note that messages currently have a maximum length of 1500 bytes, if that's too little you have to fix the corresponding literal in POE::Wheel::UDP. I don't know about your server but as problems with fragmented UDP packets seem to be rather common, datagrams are rarely made any bigger than that though---which was probably the author's idea for fixing the maximum at this value too.

    use strict; use warnings; use POE; use POE::Wheel::UDP; use POE::Filter::Stream; my $SERVER_ADDR = '127.0.0.1'; my $LOCAL_ADDR = '127.0.0.1'; my %PORTS = ( 8020 => 53036, 8019 => 53037, 8008 => 53038, 8003 => 530 +39); POE::Session->create( inline_states => { _start => sub { my ($kernel, $heap) = @_[KERNEL, HEAP]; while(my ($sport, $lport) = each %PORTS) { $heap->{"port$sport"} = POE::Wheel::UDP->new( LocalAddr => $LOCAL_ADDR, LocalPort => $lport, PeerAddr => $SERVER_ADDR, PeerPort => $sport, InputEvent => 'input', Filter => POE::Filter::Stream->new, ); $kernel->yield(heartbeat => $sport); } }, input => sub { my ($input) = $_[ARG0]; foreach my $msg (@{$input->{payload}}) { print "Message from $input->{addr}:$input->{port}: '$m +sg'\n"; } }, heartbeat => sub { my ($kernel, $heap, $sport) = @_[KERNEL, HEAP, ARG0]; $kernel->delay_add(heartbeat => 10, $sport); $heap->{"port$sport"}->put( { payload => [ "\x01" ] }); print "sent HB to $sport\n"; }, } ); POE::Kernel->run;

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://968647]
Approved by moritz
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (14)
As of 2014-12-18 20:16 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (61 votes), past polls