http://www.perlmonks.org?node_id=986570

scorpio17 has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks,

I need help writing a TCP/IP based client. I have a working client, written in C++, but I want to convert it to perl, because the ultimate goal here is to build a CGI-based web version of the client. The client is supposed to send a request to the server, then read back a response. Both client and server are running on linux (Ubuntu 10.04LTS).

The orignal client works as follows:
1) Open socket connection given host name and port.
2) Send the request length as a network ordered long integer (32-bits).

Here's how it does this:

std::string Request("abcdefghijklmnopq"); // just an example, for test +ing uint32_t msgLen = Request.length(); // msgLen = 17 for this test + case msgLen = htonl(msgLen); // convert to network ordered long int Socket().Write( reinterpret_cast< char * >(&msgLen), sizeof(msgLen));

Note: Socket() is just an accessor to the socket object (singleton).
sizeof(msgLen) = 4, since this is a 32-bit unsigned int (4 bytes).

3) Now that the server knows how much data we're going to send, go ahead and send the request:

Socket().Write( Request.begin(), Request.length() );

4) Read the length of the response:

Socket().Read(&msgLen, sizeof(msgLen)); msgLen = ntohl(msgLen); // convert back from network ordered long int Socket().Read(&buffer, msgLen); // read response into buffer

On the server side, when a request comes in, it gets processesed as follows:

char msg2[256]; memset(msg2,0,256); Socket().Read(msg2,4u); // first read 4 bytes with buffer size log.Printf("1. msg2: %04X %04X %04X %04X\n", msg2[0], msg2[1], msg2[2] +, msg2[3]); log.Printf("2. msg2: %d %d %d %d\n", msg2[0], msg2[1], msg2[2], msg2[3 +]); unsigned long nlen = *(unsigned long *)msg2; // convert char string ( +bytes) to int log.Printf("before ntohl: nlen = %d (hex: %04X)", nlen, nlen); nlen = ntohl(nlen); // convert back from network ordered long int log.Printf("after ntohl: nlen = %d (hex: %04X)", nlen, nlen); Socket().Read(msg2, nlen); // read the request log.Printf("Request: %s", msg2);

I've added the "log" statements to print output to a debugging console.

When I run the (working) C++ client, here's what I see:

New client connection accepted 1. msg2: 0000 0000 0000 0011 2. msg2: 0 0 0 17 before ntohl: nlen = 285212672 (hex: 11000000) after ntohl: nlen = 17 (hex: 0011) Request: abcdefghijklmnopq Response: hello world Closing connection.

When I run my perl version of the client, I see this:

New client connection accepted msg2: 0032 0038 0035 0032 msg2: 2 8 5 2 msg2: 50 56 53 50 before ntohl: nlen = 842348594 (hex: 32353832) after ntohl: nlen = 842544434 (hex: 32383532)

Note that the 4 bytes I'm reading into msg2 are NOT the expected '0 0 0 17', but are instead the ASCII codes for the digits '2 8 5 2', and the expected value for nlen (before ntohl) is 285212672.

So, it looks like I'm sending the data as "characters" and not as "bytes". I'm guessing maybe I should be using pack/unpack to force the data into "binary mode"???

Here is my perl code for the client:

#!/usr/bin/perl use strict; use IO::Socket; use IO::Select; $| = 1; my $host = 'localhost'; my $port = '3000'; my $data; my $socket = IO::Socket::INET->new( PeerAddr => $host, PeerPort => $port, Proto => 'tcp', Type => SOCK_STREAM, Blocking => 1, Timeout => 10, ) or die "Couldn't connect to $host:$port : $!"; my $sel = IO::Select->new($socket) or die "IO::Select error $!"; my $request = "abcdefghijklmpnoq"; my $request_length = length($request); print "Request:\n$request\n"; print "length(request) = $request_length\n"; # 17 my $x = htonl($request_length); print "x = $x\n"; # 285212672 printf("x = %x\n", $x); # 11000000 if ($sel->can_write(10)) { $socket->autoflush(1); # write request length # THIS IS WHERE I THINK SOMETHING IS GOING WRONG - THE SERVER GETS T +HE WRONG LENGTH HERE!!! $socket->send( htonl($request_length) ) or die "3a Couldn't write to + socket: $!"; # write request $socket->send( $request ) or die "3b Couldn't write to socket: $!"; if ($sel->can_read(10)) { # read response length $socket->recv($data, 4) or die "5 Couldn't read data from server: +$!"; my $reponse_length = ntohl($data); if ($sel->can_read(10)) { # read response $socket->recv($data, $reponse_length) or die "6 Couldn't read da +ta from server: $!"; print "Response:\n$data\n"; } else { print "can't read 2: $@ : $!"; } } else { print "can't read 1: $@ : $!"; } } else { print "can't write: $@ : $!"; } $socket->close(); sub htonl { my $input = shift; my $output = unpack('N*',pack('L*',$input)); return $output; } sub ntohl { my $input = shift; my $output = unpack('L*', pack('N*', $input)); return $output; }

After the two writes, it seems to get stuck, eventually timing out and dying with "Can't read 1:". But obviously, if the initial length doesn't get communicated correctly, everything downstream will malfunction.

Any help will be greatly appreciated - thanks!

update: added readmore tags

Replies are listed 'Best First'.
Re: Problem Transmitting Data via TCP/IP
by BrowserUk (Patriarch) on Aug 09, 2012 at 18:44 UTC
    and the expected value for nlen (before ntohl) is 285212672.

    Replace your (wrongly coded) htonl() with pack( 'l>', $num ). It is all you need.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    The start of some sanity?

      Thank you! I knew it had to be something simple like that.

      But what do you mean "wrongly coded"? How should htonl() be defined?

      When I change my code to this:

      # $socket->send( htonl($request_length) ) or die "3a Couldn't write + to socket: $!"; $socket->send( pack('l>',$request_length) ) or die "3a Couldn't wr +ite to socket: $!"; $socket->send( $request ) or die "3b Couldn't write to socket: $!" +;

      I get the expected output in the logging console, and the server seems to write the response to the socket, but my client fails to read the response from the socket.

      I get this message now:

      5 Couldn't read data from server:

      So something is still not quiet right... but I'm getting closer!

        But what do you mean "wrongly coded"? How should htonl() be defined?

        Your htonl:

        sub htonl { my $input = shift; my $output = unpack( 'N*', pack( 'L*', $input ) ); return $output; }
        1. First, takes the number and "packs it" (converts it to binary) as a signed long:

          Which means that the return from that first pack is 4 bytes and encoded in whatever byte-order (endianess) your current platform uses:

          $packed = pack 'l', 17;; print length $packed;; 4 print ord( substr $packed, $_, 1 ) for 0 .. 3;; 17 0 0 0

          On my intel system, that means little-endian (the low byte comes first).

        2. But then, you unpack (convert binary to ascii), those 4 bytes, treating them a big-endian long ('N'), resulting in:
          $unpacked = unpack 'N', $packed;; print length( $unpacked );; 9 print ord( substr $unpacked, $_, 1 ) for 0 .. 9;; 50 56 53 50 49 50 54 55 50 0

          Resulting in a 9-byte ascii encode string containing the number: 285212672; which is meaningless.

          htonl() could be correctly coded as sub htonl{ pack 'l>', $_[0] }; if you see the need for wrapping a built-in function in a silly named wrapper :)

        Similarly, you will need to fix your ntohl(); something like this: sub ntohl{ unpack 'l>', $_[0] } would suffice.

        That may fix the second part of your problem.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

        The start of some sanity?