Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change

Wierd File pipe problem...

by bferlin (Acolyte)
on Dec 20, 2004 at 20:47 UTC ( #416313=perlquestion: print w/replies, xml ) Need Help??

bferlin has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to make something simple that I figured would take like a few minutes in PERL and it's been causing me headaches. I can't figure out why! So I'm hoping someone with wiser eyes might recognize my malfunction. Here's what I'm doing. I'm trying to very SIMPLY take incoming serial data, mark it with the time and date, and then write it to a file. But for some unknown reason, the variable that I capture into can be PRINTED, but when I try to run any function on that variable it dies. Here's the codes I've tried:
#!/usr/bin/perl sysopen(COM, "/dev/ttyS0", O_RDONLY | O_NDELAY | O_NOCTTY) or die "can +'t open /dev/ttyS0: $!"; while(defined ($line=<COM>)) { if (length $line > 1) { #print "read[$line]\n"; system("./parse_smdr", $line); #goto breakout; } } breakout: close COM || die "COM Reader Halted";
I've also tried this:
#!/usr/bin/perl sysopen(COM, "/dev/ttyS0", O_RDONLY | O_NDELAY | O_NOCTTY) or die "can +'t open /dev/ttyS0: $!"; while ($len = sysread(COM, $buf, 16384)) { print "READ[$buf]($len)\n"; if ($len > 10) { system("./parse_smdr", $buf); } } close COM || die "COM Reader Halted";
and even:
#!/usr/bin/perl use Fcntl; use strict; my $line; sysopen(COM, "/dev/ttyS0", O_RDONLY | O_NDELAY | O_NOCTTY) or die "can +'t open /dev/ttyS0: $!"; while($line=<COM>) { if (length $line > 1) { print "read[$line]\n"; system("./parse_smdr", $line); } } close COM || die "COM Reader Halted";
And this doesn't work either....
#!/usr/bin/perl use Fcntl; use strict; my $line; open(COM, "cat /dev/ttyS0 |") or die "can't open /dev/ttyS0: $!"; while($line=<COM>) { if (length $line > 1) { print "read[$line]\n"; system("./parse_smdr", $line); } } close COM || die "COM Reader Halted";
I'm about to lose my mind. Someone help! p.s. and before you ask, the parse script was just pulled out so it didn't make me wonder where my problem was. Here's the code in case you're curious:
#!/usr/bin/perl #print "SMDR Parse Called\n"; foreach $line (@ARGV) { print "Parse[$line]\n"; ($Second, $Minute, $Hour, $Day, $Month, $Year) = localtime(tim +e); #Parse the line by column $type = substr $line,0,4; $dest = substr $line,4,6; $trunk = substr $line,10,6; $src = substr $line,16,20; $dialed = substr $line,32,9; $time = substr $line,45,5; $duration = substr $line,51,8; #print "$Hour:$Minute:$Second,"; #print "$type,$dest,$trunk,$src,$dialed,$time,$duration\n"; open(OUT,">>/home/public/SMDR/smdr$Year$month$day.pbx"); print OUT "$Hour:$Minute:$Second,$type,$dest,$trunk,$src,$dial +ed,$time,$duration\n"; close(OUT); }

Edit by BazB: add readmore tags

Replies are listed 'Best First'.
Re: Wierd File pipe problem...
by tilly (Archbishop) on Dec 20, 2004 at 21:17 UTC
    There are many things that I could comment about in your code, but I don't have time to give you a tutorial.

    However I'll suggest that the odds are very good that your bug is that you're misunderstanding the file format in some way. For instance it may be that the input you're getting doesn't use a standard return separator, so every way you have of reading it results in all of the data being put into one "line" with all of your versions of the program. Your parse program takes this data and then only reads the first line of data from it, and throws away as junk the rest of that "line" (ie the rest of your input).

    To have any hope of helping you we'd need to know more about the format of the data.

    An incidental tip. Fixed-format data formats are notoriously easy to mess up. I'd suggest that at least one of your fields have a "sanity check" that reports if it gets data that doesn't look reasonable. This will help you notice things like off-by-one errors.

Re: Wierd File pipe problem...
by runrig (Abbot) on Dec 20, 2004 at 21:47 UTC
    This line could really use some kind of or die "Error: $!" clause on the end of it. You should always check the status of your opens (and sometimes even the closes). I would also check the fields you are writing to the file and using in the open statement. I also wonder about why you have the parse/write code in a different executable than the read code.

    Update: tilly's right, fixed length formats are hard to get right (though it's hard to tell if that's even the problem here). That's why I sometimes like to use Parse::FixedLength and its debug option even though I generally might not use it when there are not that many fields (I would first try to figure out what your problem is though, or else you may end up with another problem -- figuring out another module).

Re: Wierd File pipe problem...
by KeighleHawk (Scribe) on Dec 20, 2004 at 22:18 UTC
    first, for cleanliness, I think unpack would be nicer than all the substrings.

    second, does $line have a carraige return stuck on it? I don't see you chomp'ing it. Some times that causes things to misfire in bizarre ways; especially if you are calling out to the shell with it...

    third, check your return code on your system call. That may produce something useful...

      I'll give it a try, however the first time I wrote this, the second script was inside the first and it does the same thing. I only seperated them to help with troubleshooting...
Re: Wierd File pipe problem...
by runrig (Abbot) on Dec 20, 2004 at 23:54 UTC
    Please provide a complete example. Put your parsing code back with your reading code, because your just creating more places for your program to go wrong. Check the status of your open as I said before. Provide some sample input and put it in a DATA section, like the following (the following is pseudo code, please fill in with actual code):
    while (<DATA>) { #parse data #open output file #write data to output file } # close output file __DATA__ #some sample input
    Once you turn the above into real code, if you are still having problems with it, then ask again. If you get the above working, then replace a bit at a time (e.g. open your real input source -- maybe it's the problem, and I don't know why you're using sysopen instead of open) until you get your desired result.
      I'll give that a try, however if you check my scratchpad (bferlin's scratchpad) you'll notice I have like three or four versions using open, not using open, I've tried lots of things...
Re: Wierd File pipe problem...
by steves (Curate) on Dec 20, 2004 at 21:58 UTC

    Where exactly is it dying? Can you show us the specific line of code it's dying on, along with some output up to that point?

      see, it's not really Dying. The problem is that it's NOT dying. It works great to a point. It runs. It sits there, and it prints out that it's finding 'LINE1' then it waits, then 'LINE2' ... Because in my code you see I have it printing the line that it pulls before passing it on.
      print "read[$line]\n";
      Is in the read_smdr. The problem is, if I, for instance, chop the data, the return is empty. Even though if I print $line, it shows the full 80 character line. If I pass it to another program, as I have in the code here:
      system("./parse_smdr", $line);
      the new program reports from it's args the first line perfect: It'll say:
      from the command in parse_smdr that says:
      print "Parse[$line]\n";
      Like it's supposed to. Then the second iteration of READ, it'll pass empty string. No matter what. Even if you just run a standard perl command, the command returns as if it was passed an empty variable.... it's VERY bizzarre. Mind you, that the FIRST TIME the while loop in read_smdr runs it works PERFECT...

        It would help to see the exact code and output from a failure. I still don't understand 100% but it sounds like your failure is on the read side -- that you try reading from the device and get an empty "line". In the first read examples, you're opening the device with O_NDELAY. If I remember correctly, on most *nix systems, the way that behaves with a terminal device is that it returns nothing if nothing is available on the terminal at the time of the read. It seems like that behavior may be matching exactly what you're seeing. I also believe that an open using O_NDELAY may itself return immediately, before the terminal device is ready, which could cause problems that only show up when timing changes.

        In general, reading directly from terminal devices like this is tricky and error prone. So you may want to revisit exactly what it is you're trying to achieve -- something else I can't really gather from the example so far. The last open is cat'ing the terminal device and reading from that output. That should do almost the same thing the opens before it are attempting, but it should avoid the O_NDELAY issues ... I think. It would help to know the context the terminal device is bneing read in -- what's being typed there, how that device relates to the one the program is running on, etc.

Re: Wierd File pipe problem...
by zentara (Archbishop) on Dec 21, 2004 at 14:42 UTC
    Maybe you are misinterpreting how a /dev/file outputs? I've seen this sort of behavior before, where the filehandle dosn't close, but output stops. I needed to rewind the filehandle to the beginning, after each read. Try putting this in your read loop.
    seek COM, 0, SEEK_SET or die "Cannot rewind $!"; + truncate COM, 0 or die "Cannot truncate $!":
    You may, or may not, need the truncate.

    I'm not really a human, but I play one on earth. flash japh
      It returns "cannot rewind, illegal seek" with the open(COM, "cat /dev/ttyS0 |". I thought I was using the file handle correctly. Should I be trying to open the file directly instead of just catting the characters off of it. I started here because I figured it was the cleanest, using CAT to do my dirty work for me. Thanks for the idea though... any more?
        Thats what I would try:
        open(COM,"< /dev/ttyS0")
        Then try the rewind. I have a few examples of reading serial ports that seem to work on linux, if you want I could mail them off to you. You might want to look at Device::SerialPort;

        Here is a nice explanation-script I found once on the net.

        #!/usr/bin/perl + + + + #the ubiquitous RS-232 Serial Port. built an interface + + #for my Aware Electronics Geiger Counter (RM-70) that + + #used a Basic Stamp IISX chip set. Every 20 seconds it + + #would spit out an ASCII string of radiation and + + #temperature data. I had hunted around for examples + + #using the Device::SerialPort module and found many. + + #Most were copies of code used to read PBX data. But + + #it wouldn't work. Ouch. Finally, after much research + + #I realized that - by golly - you had to terminate + + #the IO with a new line and not a carriage return. + + #Oddly doing a cat </dev/ttyS0 worked, which faked me + + #out. So I added the line in the tty setup below that + + #converts CR to NL and zap-ity-do-dah it started terminating + + #and sending each read. Long live RS-232. Please send + + #along any suggestions and improvements. This also probably + + #explains why I had failed to get good reads from a + + #cheap-o RS-232 capable DVM a few years ago. + + + + #This was done in Perl 5.8.0 under RH Linux 9.0. + + + + #file + + # + + # Author: David Drake + + # Date: July 18, 2003 + + # Requirements: Device::SerialPort 0.22 (from cpan July 2003) + + # + + # Version: 1.0 + + #This script is used to read a serial port to obtain data from a + + #combined Geiger counter and temperature sensor. + + #The Geiger Counter is an Aware Electronics RM-70 unit. Each count + + #maps to one microR per hour. The RM-70 pulse output is sent to a + + #Basic Stamp-IISX microcontroller. The BS2SX accumulates counts for 20 + + #seconds and then sends a serial data stream out of its serial port. + + #The data stream goes into the input serial port on the Linux system. + + #This program then tabulates the data to a log file. + + + + use Device::SerialPort; + + use Time::gmtime; + + + + $LOGDIR = "/home/zentara/perlplay/serial-comm"; # path to data fil +e + $LOGFILE = "geiger.log"; # file name to output to + + $PORT = "/dev/ttyS1"; # port to watch + + + + # + + # + + # Serial Settings + + # + + + + #make the serial port object + + #note the need to convert carriage returns to new lines to terminate e +ach + #read. + + + + $ob = Device::SerialPort->new($PORT) || die "Can't Open $PORT: $!"; + + + + $ob->baudrate(9600) || die "failed setting baudrate"; + + $ob->parity("none") || die "failed setting parity"; + + $ob->databits(8) || die "failed setting databits"; + + $ob->stty_icrnl(1) || die "failed setting convert cr to new line"; + + $ob->handshake("none") || die "failed setting handshake"; + + + + $ob->write_settings || die "no settings"; + + + + # + + # open the logfile, and Port + + # open( LOG, ">>${LOGDIR}/${LOGFILE}" ) + + || die "can't open smdr file $LOGDIR/$LOGFILE for append: $SUB $!\n" +; + + + select(LOG), $| = 1; # set nonbuffered mode, gets the chars out NOW + + + + open( DEV, "<$PORT" ) || die "Cannot open $PORT: $_"; + + + + # + + # Loop forver, logging data to the log file + + # + + + + while ( $_ = <DEV> ) { # print input device to file + + $gmc = gmctime(); + + print LOG $gmc, " ", $_; + + } + + + + undef $ob; + + + + #we are done dude

        I'm not really a human, but I play one on earth. flash japh

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://416313]
Approved by kutsu
Front-paged by kutsu
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (4)
As of 2019-10-19 19:25 GMT
Find Nodes?
    Voting Booth?