Beefy Boxes and Bandwidth Generously Provided by pair Networks
good chemistry is complicated,
and a little bit messy -LW
 
PerlMonks  

Unknown characters

by HyperDevil (Initiate)
on Feb 08, 2011 at 22:17 UTC ( #887070=perlquestion: print w/replies, xml ) Need Help??

HyperDevil has asked for the wisdom of the Perl Monks concerning the following question:

I am programming an AIVDM parser in Perl.
It receives data using UDP (recv), the mechanism seems to work fine.
But the issue is that there are some unknown characters in the message which perl puts into the scalar variable which i cannot seem to get rid of.
I have tried to chomp the scalar which removes the text completely, but length() still reports correctly.
A regex which deletes all hex values which are not numbers or letters, with no effect.
Converted the scalar to hex and back into a another scalar, the new scalar has the same issue.

Code:
while ($sock->recv($msg, 1024)) { $length = length($msg); print "MESSAGE: $msg LENGTH: $length\n"; }
Output is:
MESSAGE: !A LENGTH: 2 MESSAGE: IVDM,1,1,,A,?3nGtL0uPi7pD00,2*46 LENGTH: 34
The first line is ok!
Did you see the second line, where does the space come from?
There is also no newline character in the $msg.

Code:
while ($sock->recv($msg, 1024)) { $length = length($msg); print "MESSAGE: $msg LENGTH: $length"; }
Now without the \n i get this:
MESSAGE: !AI LENGTH: 3MESSAGE: ADM,1,1,,A,13mK@M0P000gVvvTC=4:LgwV2L0B +,0*4C LENGTH: 46MESSAGE: !AIVDM,1,1,,A,40 LENGTH: 16MESSAGE: 2M43AudTF;o0fr +sPTBHl700L0h,0*5A
Now the second message i receive make a new line without me specifieng a \n.

I have been trying to remove these weird characters for days without avail, so i need some help from the experts :)
Update:
while ($sock->recv($msg, 1024)) { $msg1 = substr $msg, 0, -1; print "MESSAGE: $msg1 LENGTH: $length"; }
Does also not give any output at all!

Replies are listed 'Best First'.
Re: Unknown characters
by mvaline (Friar) on Feb 08, 2011 at 23:58 UTC

    I suspect an encoding problem. Is the socket operating on bytes or characters? Can you view a hexdump of the source lines to check what bytes you're actually receiving? I believe the AIVDM payload is an ASCII-encoded bit vector. By default, sockets operate on bytes. Try using the :encoding pragma when you open the socket to set it to operate on characters.

Re: Unknown characters
by umasuresh (Hermit) on Feb 08, 2011 at 22:21 UTC
    try
     chomp $msg;
    before printing!
      I have tried chomp; what happens is when i then try to print the string it is empty.
      When i length($msg); it still gives my a 46 back.
        Did you remember to $|=1? Otherwise the reason you're not seeing any output at all is probably just buffering.
Re: Unknown characters
by rowdog (Curate) on Feb 10, 2011 at 05:43 UTC

    I suspect you're running into the Internet Line Terminator aka \r\n. Try something like this.

    { local $/ = "\r\n"; chomp $msg; }

    Better yet, check what that character actually is. \r = 13.

    chomp $msg; print ord(reverse $msg), "\n";

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://887070]
Front-paged by Arunbear
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (3)
As of 2021-12-01 11:06 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    R or B?



    Results (3 votes). Check out past polls.

    Notices?