Re^4: Parsing Data logger files ...

[RedTussock]

Something like

#!/usr/bin/perl
use warnings;
use strict;

$/ = "\n1=";
while (<>) {
    my @fields = split /\n/;
    s/^[1-6]=// for @fields;
      open (MYFILE, '>/a/b/c/perl/dl_parser/logData.txt');
      print MYFILE "+(join ',', @fields[0 .. 5]), "\n"";
      close (MYFILE); 
    
}

[Kenosis]

Consider the following:

#!usr/bin/perl
use strict;
use warnings;

$/ = "\n1=";

open my $fh, '>', '/a/b/c/perl/dl_parser/logData.txt' or die $!;

while (<>) {
    my @fields = split /\n/;
    s/^[1-6]=// and s/\s+$// for @fields;
    print $fh ( join ',', map qq/"$_"/, @fields[ 0 .. 5 ] ) . "\n";
}

close $fh;

For each csv line, your script will open, write to, and then close the logData.txt file. When done, that file will have only one line, viz., the last one written to it. The above will write all csv lines to your file, since the open and close statements are outside the while loop. Additionally, note that open is using a lexically-scoped file handle (my $fh) and that it is a three-argument open.

Have added two items to the script. The first is an additional substitution, since there was a space at the end of the last three fields. The second, a map statement, encloses each field within quotations, since the first data field contains a comma (remove the map qq/"$_"/, if you want the date and time to be treated as separate csv fields).

Hope this helps!

[RedTussock]

Yep OK, Sorry for the delay I had to pick family up of the plane ... The code offered creates afile indeed it does ... but It chokes on multiple files I think ... I get this result

Smitty@smittytech:~/scripts/perl/dl_parser> perl lineJumper.pl /home/S
+mitty/scripts/perl/dl_parser/down/*
Use of uninitialized value in join or string at lineJumper.pl line 12,
+ <> chunk 478.
Use of uninitialized value in join or string at lineJumper.pl line 12,
+ <> chunk 478.
Use of uninitialized value in join or string at lineJumper.pl line 12,
+ <> chunk 478.
Use of uninitialized value in join or string at lineJumper.pl line 12,
+ <> chunk 478.

which I think is the script not liking multiple files to parse ?? ..

[kcott]

G'day RedTussock,

Welcome to the monastery.

I'm reasonably certain that your problem is in map qq/"$_"/, @fields[ 0 .. 5 ] and is probably caused by @fields not containing 6 elements. Here's some troubleshooting hints.

Check the files you're reading (i.e. ls -l /home/Smitty/scripts/perl/dl_parser/down/*). Are all of these the files you want? Perhaps use *.txt (or similar) to limit the files you're processing.

Do all of the files have the format you're expecting? You may be able to check this by inspection. If not, add some debugging code around while (<>) { like this:

my $last_file = '';
while (<>) {
    print "$ARGV\n" if $ARGV ne $last_file;
    $last_file = $ARGV;

[perlop - I/O Operators explains <> and $ARGV]

You should see something like:

some_file_name
another_file_name
Use of uninitialized value in ...
different_file_name
Use of uninitialized value in ...

You'll now know which files are causing the problems and you can focus your investigation accordingly.