Parsing/regex question

vxp has asked for the wisdom of the Perl Monks concerning the following question:

I'm parsing input and placing stuff that I am parsing into variables, so I can mangle them and do some logical operations on them

The file is read line-by-line and a bunch of regexes parse the relevant bits.

My problem comes in the form of "Position/Spot/Mark has rolled into ...." lines seen below in the script's output. Since input is read line by line, I can't craft a multiline regex to create a "$position_rolled_into = <datestamp>" for me. I was wondering if anyone can suggest a solution to this.

Thanks!

=========== DATE INITIALIZATION ===========
Today is: 07/06/2009
Tomorrow is: 07/07/2009
Yesterday is: 07/05/2009

=========== INPUT ===========
table_name   keyid        description
------------ ------------ ----------------------------------------
COCARRY      StoreToDDI   DONE
COCARRY      rundate      7/3/2009
ROLL         ignore_local
ROLL         mark         DONE     07/03/2009 22:09
ROLL         position     DONE     07/03/2009 22:08
ROLL         rundate      07/03/2009
ROLL         spot         DONE     07/03/2009 22:08
SPLAdj       data         DONE Jul  4 2009 12:51AM
SPLAdj       lastcompdate Jul  3 2009 12:00AM
SPLAdj       rundate      Jul  3 2009 12:00AM
SPLbatch     data         DONE Jul  3 2009 11:55PM
SPLbatch     lastcompdate Jul  3 2009 12:00AM
SPLbatch     rundate      Jul  3 2009 12:00AM
SPLbatchNew  data         DONE Jul  4 2009 12:50AM
SPLbatchNew  lastcompdate Jul  3 2009 12:00AM
SPLbatchNew  rundate      Jul  3 2009 12:00AM
(16 rows affected)
Position has rolled into
--------------------------
Jul  6 2009 12:00AM
(1 row affected)
Spot has rolled into
--------------------------
Jul  6 2009 12:00AM
(1 row affected)
Mark has rolled into
--------------------------
Jul  6 2009 12:00AM
(1 row affected)

=========== AFTER PARSING ===========
-- COCARRY StoreToDDI: DONE
-- COCARRY rundate: 7/3/2009
-- ROLL ignore_local: nothing
-- ROLL mark desc: DONE - 07/03/2009 - 22:09
-- ROLL position: DONE - 07/03/2009 - 22:08
-- ROLL rundate: 07/03/2009
-- ROLL spot desc: DONE - 07/03/2009 - 22:08
-- SPLAdj data desc: DONE - Jul  4 2009 - 12:51 - AM
-- SPLAdj lastcompdate desc: Jul  3 2009 - 12:00 - AM
-- SPLAdj rundate desc: Jul  3 2009 - 12:00 - AM
-- SPLbatch data desc: DONE - Jul  3 2009 - 11:55 - PM
-- SPLbatch lastcompdate desc: Jul  3 2009 - 12:00 - AM
-- SPLbatch rundate desc: Jul  3 2009 - 12:00 - AM
-- SPLbatchNew data desc: DONE - Jul  4 2009 - 12:50 - AM
-- SPLbatchNew lastcompdate desc: Jul  3 2009 - 12:00 - AM
-- SPLbatchNew rundate desc: Jul  3 2009 - 12:00 - AM
[download]

Full code below.

#!/usr/bin/perl

my $tmp_dir = "./tmp";
my $region = shift;

###################################################
# is the region correct? was it even given?
###################################################
$region = lc($region);
unless (($region =~ /^ny$/) || ($region =~ /^ln$/) || ($region =~ /^tk
+$/) || ($region =~ /^hk$/) || ($region =~ /^se$/) || ($region =~ /^mo
+$/) ) {
        print "The acceptable regions are:\nNY\nLN\nTK\nHK\nSE\nMO\n";
        exit(-1);
}

print "=========== DATE INITIALIZATION ===========\n";

############################################
# get today's date in the mm/dd/yyyy format
############################################
my ($sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst) = loc
+altime (time);
$year = 1900 + $year;
$mon = $mon + 1;

if ($mon =~ /(\d)/) {
        $mon = "0" . $1;
}
if ($mday =~ /(\d)/) {
        $mday = "0" . $1;
}

my $today = $mon . "/" . $mday . "/" . $year;
print "Today is: $today\n";

###############################################
# get tomorrow's date in the mm/dd/yyyy format
###############################################
($sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst) = localt
+ime (time + 86400);
$year = 1900 + $year;
$mon = $mon + 1;

if ($mon =~ /(\d)/) {
        $mon = "0" . $1;
}
if ($mday =~ /(\d)/) {
        $mday = "0" . $1;
}

my $tomorrow = $mon . "/" . $mday . "/" . $year;
print "Tomorrow is: $tomorrow\n";

###############################################
# get yesterday's date in the mm/dd/yyyy format
###############################################
($sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst) = localt
+ime (time - 86400);
$year = 1900 + $year;
$mon = $mon + 1;

if ($mon =~ /(\d)/) {
        $mon = "0" . $1;
}
if ($mday =~ /(\d)/) {
        $mday = "0" . $1;
}

my $yesterday = $mon . "/" . $mday . "/" . $year;
print "Yesterday is: $yesterday\n";

################################################################
# process the input. parse it, place everything into variables.
################################################################

open(MSOUT, "$tmp_dir/$region") or die "Couldn't open $tmp_dir/$region
+: $!\n";

print "\n=========== INPUT ===========\n";

while (<MSOUT>) {
   s/^\s+//;
   s/\s+$//;
   my $line = $_;
   chomp($line);
   next unless length > 0;
   print "$line\n";

   #
   #   COCARRY      StoreToDDI   DONE
   #
   if ($line =~ /^COCARRY\s+StoreToDDI\s+(.*)/) {
    $COCARRY_StoreToDDI_desc = $1;
   }
   
   #
   #   COCARRY      rundate      7/3/2009
   #
   elsif ($line =~ /^COCARRY\s+rundate\s+(.*)/) {
     $COCARRY_rundate_desc = $1;
   }
  
   #
   #   ROLL         ignore_local
   #
   elsif ($line =~ /^ROLL\s+ignore_local$/) {
         $ROLL_ignore_local_desc = "nothing";
   }

   #
   #   ROLL         ignore_local
   #
   elsif ($line =~ /^ROLL\s+ignore_local\s+(.*)/) {
         $ROLL_ignore_local_desc = $1;
   }

   #
   #   ROLL         mark         DONE     07/03/2009 22:09
   #
   elsif ($line =~ /^ROLL\s+mark\s+(\w+)\s+(\d+\/\d+\/\d+)\s(\d+:\d+)/
+) {
         $ROLL_mark_desc = $1;
     $ROLL_mark_desc_date = $2;
     $ROLL_mark_desc_time = $3;
   }
   
   #
   #   ROLL         position     DONE     07/03/2009 22:08
   #
   elsif ($line =~ /^ROLL\s+position\s+(\w+)\s+(\d+\/\d+\/\d+)\s(\d+:\
+d+)/) {
         $ROLL_position_desc = $1;
     $ROLL_position_desc_date = $2;
     $ROLL_position_desc_time = $3;
   }

   #
   #   ROLL         rundate      07/03/2009
   #
   elsif ($line =~ /^ROLL\s+rundate\s+(.*)/) {
         $ROLL_rundate_desc = $1;
   }

   #
   #   ROLL         spot         DONE     07/03/2009 22:08
   #
   elsif ($line =~ /^ROLL\s+spot\s+(\w+)\s+(\d+\/\d+\/\d+)\s(\d+:\d+)/
+) {
         $ROLL_spot_desc = $1;
         $ROLL_spot_desc_date = $2;
         $ROLL_spot_desc_time = $3;
   }

   #
   #   SPLAdj       data         DONE Jul  4 2009 12:51AM
   #
   elsif ($line =~ /^SPLAdj\s+data\s+(\w+)\s+(\w+\s+\d+\s\d+)\s(\d+:\d
++)(\w+)/) {
         $SPLAdj_data_desc = $1;
         $SPLAdj_data_desc_date = $2;
         $SPLAdj_data_desc_time = $3;
     $SPLAdj_data_desc_ampm = $4;
   }
   
   #
   #   SPLAdj       lastcompdate Jul  3 2009 12:00AM
   #
   elsif ($line =~ /^SPLAdj\s+lastcompdate\s+(\w+\s+\d+\s\d+)\s(\d+:\d
++)(\w+)/) {
         $SPLAdj_lastcompdate_desc_date = $1;
         $SPLAdj_lastcompdate_desc_time = $2;
     $SPLAdj_lastcompdate_desc_ampm = $3;
   }

   #
   #   SPLAdj       rundate      Jul  3 2009 12:00AM
   #
   elsif ($line =~ /^SPLAdj\s+rundate\s+(\w+\s+\d+\s\d+)\s(\d+:\d+)(\w
++)/) {
         $SPLAdj_rundate_desc_date = $1;
         $SPLAdj_rundate_desc_time = $2;
         $SPLAdj_rundate_desc_ampm = $3;
   }
  
   #
   #   SPLbatch     data         DONE Jul  3 2009 11:55PM
   #
   elsif ($line =~ /^SPLbatch\s+data\s+(\w+)\s+(\w+\s+\d+\s\d+)\s(\d+:
+\d+)(\w+)/) {
         $SPLbatch_data_desc = $1;
         $SPLbatch_data_desc_date = $2;
         $SPLbatch_data_desc_time = $3;
         $SPLbatch_data_desc_ampm = $4;
   }

   #
   #   SPLbatch     lastcompdate Jul  3 2009 12:00AM
   #
   elsif ($line =~ /^SPLbatch\s+lastcompdate\s+(\w+\s+\d+\s\d+)\s(\d+:
+\d+)(\w+)/) {
         $SPLbatch_lastcompdate_desc_date = $1;
         $SPLbatch_lastcompdate_desc_time = $2;
         $SPLbatch_lastcompdate_desc_ampm = $3;
   }

   #
   #   SPLbatch     rundate      Jul  3 2009 12:00AM
   #
   elsif ($line =~ /^SPLbatch\s+rundate\s+(\w+\s+\d+\s\d+)\s(\d+:\d+)(
+\w+)/) {
         $SPLbatch_rundate_desc_date = $1;
         $SPLbatch_rundate_desc_time = $2;
         $SPLbatch_rundate_desc_ampm = $3;
   }

   #
   #   SPLbatchNew  data         DONE Jul  4 2009 12:50AM
   #
   elsif ($line =~ /^SPLbatchNew\s+data\s+(\w+)\s+(\w+\s+\d+\s\d+)\s(\
+d+:\d+)(\w+)/) {
         $SPLbatchNew_data_desc = $1;
         $SPLbatchNew_data_desc_date = $2;
         $SPLbatchNew_data_desc_time = $3;
         $SPLbatchNew_data_desc_ampm = $4;
   }


   #
   #   SPLbatchNew  lastcompdate Jul  3 2009 12:00AM
   #
   elsif ($line =~ /^SPLbatchNew\s+lastcompdate\s+(\w+\s+\d+\s\d+)\s(\
+d+:\d+)(\w+)/) {
         $SPLbatchNew_lastcompdate_desc_date = $1;
         $SPLbatchNew_lastcompdate_desc_time = $2;
         $SPLbatchNew_lastcompdate_desc_ampm = $3;
   }


   #
   #   SPLbatchNew  rundate      Jul  3 2009 12:00AM
   #
   elsif ($line =~ /^SPLbatchNew\s+rundate\s+(\w+\s+\d+\s\d+)\s(\d+:\d
++)(\w+)/) {
         $SPLbatchNew_rundate_desc_date = $1;
         $SPLbatchNew_rundate_desc_time = $2;
         $SPLbatchNew_rundate_desc_ampm = $3;
   }

}
close(MSOUT);


print "\n=========== AFTER PARSING ===========\n";
print "-- COCARRY StoreToDDI: $COCARRY_StoreToDDI_desc\n";    
print "-- COCARRY rundate: $COCARRY_rundate_desc\n";
print "-- ROLL ignore_local: $ROLL_ignore_local_desc\n";
print "-- ROLL mark desc: $ROLL_mark_desc - $ROLL_mark_desc_date - $RO
+LL_mark_desc_time\n";
print "-- ROLL position: $ROLL_position_desc - $ROLL_position_desc_dat
+e - $ROLL_position_desc_time\n";
print "-- ROLL rundate: $ROLL_rundate_desc\n";
print "-- ROLL spot desc: $ROLL_spot_desc - $ROLL_spot_desc_date - $RO
+LL_spot_desc_time\n";
print "-- SPLAdj data desc: $SPLAdj_data_desc - $SPLAdj_data_desc_date
+ - $SPLAdj_data_desc_time - $SPLAdj_data_desc_ampm\n";
print "-- SPLAdj lastcompdate desc: $SPLAdj_lastcompdate_desc_date - $
+SPLAdj_lastcompdate_desc_time - $SPLAdj_lastcompdate_desc_ampm\n";
print "-- SPLAdj rundate desc: $SPLAdj_rundate_desc_date - $SPLAdj_run
+date_desc_time - $SPLAdj_rundate_desc_ampm\n";
print "-- SPLbatch data desc: $SPLbatch_data_desc - $SPLbatch_data_des
+c_date - $SPLbatch_data_desc_time - $SPLbatch_data_desc_ampm\n";
print "-- SPLbatch lastcompdate desc: $SPLbatch_lastcompdate_desc_date
+ - $SPLbatch_lastcompdate_desc_time - $SPLbatch_lastcompdate_desc_amp
+m\n";
print "-- SPLbatch rundate desc: $SPLbatch_rundate_desc_date - $SPLbat
+ch_rundate_desc_time - $SPLbatch_rundate_desc_ampm\n";
print "-- SPLbatchNew data desc: $SPLbatchNew_data_desc - $SPLbatchNew
+_data_desc_date - $SPLbatchNew_data_desc_time - $SPLbatchNew_data_des
+c_ampm\n";
print "-- SPLbatchNew lastcompdate desc: $SPLbatchNew_lastcompdate_des
+c_date - $SPLbatchNew_lastcompdate_desc_time - $SPLbatchNew_lastcompd
+ate_desc_ampm\n";
print "-- SPLbatchNew rundate desc: $SPLbatchNew_rundate_desc_date - $
+SPLbatchNew_rundate_desc_time - $SPLbatchNew_rundate_desc_ampm\n";
[download]

Comment on Parsing/regex question Select or Download Code

Replies are listed 'Best First'.
Re: Parsing/regex question by ikegami (Patriarch) on Jul 06, 2009 at 18:37 UTC
Basically, `my $position_rolled = 0; while (<$fh>) { chomp; if ($position_rolled) { $position_rolled = 0; $position_rolled_into = $_; } elsif ($_ eq 'Position has rolled into') { $position_rolled = 1; } ... }` [download]	[reply] [d/l]
Re: Parsing/regex question by superfrink (Curate) on Jul 06, 2009 at 18:39 UTC
What do you want the script to do when it encounters a "has rolled into" line? eg: skip the line and the following line or two? You could set a variable used to keep state when a line matches and at the top of the loop skip lines if the variable is set. Here is some sample completely un-tested code: `my $lines_to_skip = 0; LINE: while (<MSOUT>) { if (0 < $lines_to_skip) { $lines_to_skip --; next LINE; } ... if ($line =~ /^(Position\|Spot\|Mark) has rolled into$/)) { $lines_to_skip = 1; next LINE; } }` [download]	[reply] [d/l]
Re^2: Parsing/regex question by vxp (Pilgrim) on Jul 06, 2009 at 18:43 UTC
When the script encounters a "Position has rolled into" line it should skip everything up to the datestamp, and place that datestamp into a $position_rolled_into var Same goes for the "spot/mark has rolled into" :)	[reply]
Re^3: Parsing/regex question by ikegami (Patriarch) on Jul 06, 2009 at 19:21 UTC
should skip everything up to the datestamp oops, missed the "---" line. Adding one line of code addresses that: `my $position_rolled = 0; while (<$fh>) { chomp; if ($position_rolled) { next if !/^... .\d 20\d\d/; $position_rolled = 0; $position_rolled_into = $_; } elsif ($_ eq 'Position has rolled into') { $position_rolled = 1; } ... }` [download]	[reply] [d/l]
Re^3: Parsing/regex question by superfrink (Curate) on Jul 06, 2009 at 19:03 UTC
Then in the while loop you can do something like this: `... if (0 < $lines_to_skip) { if ($line =~ /datestamp-regex/) { next LINE; } # we are at the next date stamp line so fall through, etc. } ...` [download] Also rename `$lines_to_skip` to something suitable.	[reply] [d/l] [select]
Re: Parsing/regex question by vxp (Pilgrim) on Jul 06, 2009 at 19:34 UTC
Thanks guys! I ended up doing the following: Right before the while (<MSOUT>) line: `my $position_rolled = 0; my $spot_rolled = 0; my $mark_rolled = 0;` [download] inside the while (<MSOUT>) loop: `... next if $line =~ /^-/; if ($position_rolled) { $position_rolled = 0; $position_rolled_into = $line; } if ($spot_rolled) { $spot_rolled = 0; $spot_rolled_into = $line; } if ($mark_rolled) { $mark_rolled = 0; $mark_rolled_into = $line; } ... if ($line =~ /Position has rolled into/) { $position_rolled = 1; } if ($line =~ /Spot has rolled into/) { $spot_rolled = 1; } if ($line =~ /Mark has rolled into/) { $mark_rolled = 1; }` [download] And then printing it out: `print "Position rolled into: $position_rolled_into\n"; print "Spot rolled into: $spot_rolled_into\n"; print "Mark rolled into: $mark_rolled_into\n";` [download] Works like a charm :)	[reply] [d/l] [select]
Re: Parsing/regex question by toolic (Bishop) on Jul 06, 2009 at 19:45 UTC
Unrelated to your problem ... here is a suggestion to simplify your code. Use POSIX to determine your dates: `print "=========== DATE INITIALIZATION ===========\n"; ############################################ # get dates in the mm/dd/yyyy format ############################################ use POSIX; print "Today is: " , strftime('%m/%d/%Y', localtime), "\n"; print "Tomorrow is: " , strftime('%m/%d/%Y', localtime(time + 86400)) +, "\n"; print "Yesterday is: " , strftime('%m/%d/%Y', localtime(time - 86400)) +, "\n";` [download]	[reply] [d/l]
Re^2: Parsing/regex question by vxp (Pilgrim) on Jul 06, 2009 at 19:58 UTC
incorporated into my code, less lines now :D thanks for the tip!	[reply]
Re^3: Parsing/regex question by ikegami (Patriarch) on Jul 06, 2009 at 20:11 UTC
Don't, it's wrong. 86400 seconds sooner/later can be the same day. 86400 seconds sooner/later can be two days earlier/later. `use DateTime qw( ); my $today = DateTime->today( time_zone => 'local' ); my $tomorrow = $today->add ( days => 1 ); my $yesterday = $today->subtract( days => 1 ); print "Today is: ", $today ->strftime('%m/%d/%Y'), "\n"; print "Tomorrow is: ", $tomorrow ->strftime('%m/%d/%Y'), "\n"; print "Yesterday is: ", $yesterday->strftime('%m/%d/%Y'), "\n";` [download] Update: Added solution. Update: Fixed spelling of `time_zone`. (underscore was missing)	[reply] [d/l] [select]
Re^4: Parsing/regex question by toolic (Bishop) on Jul 06, 2009 at 20:21 UTC
Re^4: Parsing/regex question by vxp (Pilgrim) on Jul 07, 2009 at 18:02 UTC
Re^5: Parsing/regex question by ikegami (Patriarch) on Jul 07, 2009 at 18:13 UTC
Some notes below your chosen depth have not been shown here


Syntactic Confectionery Delight
	PerlMonks