Processing unpredictable, asynchronous input from a Net::Telnet connection

[Hercynium]

Fellow Monks,

I've reached an interesting question while designing a module to facilitate collecting and parsing TL1 output, though not related to TL1 itself, but rather to the behavior of Net::Telnet.

Disclaimer: I'm not stricken with NIH syndrome. I've evaluated both Net::TL1 and the non-CPAN TL1 modules (code and functionality) and neither seems to fit my needs.

My question requires a smattering of context. The TL1 data I am interested in is sent autonomously, over a telnet-like session. There is no log-in sequence. I simply connect and begin reading lines of input as they come, sometimes in torrents, sometimes nothing for minutes at a time.

The input is composed of multi-line 'messages' terminated by a semi-colon, on it's own line. Since the parsing of these is not important right now, I won't describe it any further.

What *is* important is that I am able to avoid losing *any* of these messages. Sometimes they come in, hundreds at a time, and other times there are none for minutes at a time. I don't like the idea of just looping forever, waiting for input and ignoring timeouts... but maybe I'm just being foolish :)

The question: I am curious as to how Net::Telnet collects input over the socket - if my program is busy processing a message and several new ones come in during that time, will I lose any before I get to the next readline()? Does Net::Telnet use select() and/or an internal buffer behind the scenes? Do I need to worry about the size of that buffer?

I must admit, I've already written a (rather ugly) module to solve this problem by using a thread to handle the reading and parsing of lines from the TL1 connection, which are then pushed onto a shared array for processing. It works well, but I'm not convinced it's the simplest solution, and the error-handling and recovery routines are the ugliest I've ever written! I am hunting for advice that will help me make this CPAN-worthy, and hopefully a lot simpler and more robust than I think it is right now.

I am a bit daunted by the code in Net::Telnet and what I('m looking for does not seem to be covered in the documentation or even elsewhere on the internet. I've poked around with select() and IO::Socket and IO::Select but I don't see the point in using those if Net::Telnet can handle the job for me.

[laceytech]

From the Net::Telnet docs
Other reasons to use this module than strictly with a TELNET port are:
* You're not familiar with sockets and you want a simple way to make client connections to TCP services.
* You want to be able to specify your own time-out while connecting, reading, or writing.
* You're communicating with an interactive program at the other end of some socket or pipe and you want to wait for certain patterns to appear.

max_buffer_length allows you to set the input buffer size, default is 1MB
timeout can be set to undefined to turn off timeout

I have found the documentation to be good and the module to be flexible

Jim

[Hercynium]

Some experimentation with different ways of getting the input has shown me that Net::Telnet seems to be able to buffer the data coming in over the socket sufficiently that I haven't lost any in my tests. I guess I just needed someone else to reassure me that it would work.

However, I am still curious as to *how* it works...

What I've got so far is this: The input is data sent from the remote host and the output is the data my app sends to the remote host. My app uses Net::Telnet to retrieve the input lines and send the output lines. The documentation says that all output is flushed while all input is buffered. What I don't understand is how the buffering happens - is that something a socket does behind-the-scenes? Because if Net::Telnet is doing the buffering I want to understand how the code is executing - I *know* that input is coming in while my app is processing, between calls to Net::Telnet's readline() method.

Still, my second shot at this module is shaping up to be much, much better code without the threads.

[fmerges]

Hi,

I recommend you this book UNIX Network Programming

Regards,