http://www.perlmonks.org?node_id=556770

sunadmn has asked for the wisdom of the Perl Monks concerning the following question:

Hey all I have what should be simple I hope to do. Below I have a bit of code that looks at a news servers stats file and adds some data up and inserts into a DB. This works fine as is but I have come to the conclusion that my data manipulation needs to be a bit fancier. The problem is this script is ran out of cron every four hours (six times daily), but when we reach the 23:59:59 hour the data could be 100% today 0% tomorrow or the exact opposite. With that said I was thinking I could use some sort of hash to take the unixtime stamp inside the file and store all of todays data in today and all of tomorrow in tomorrow. The problem is how do I store that data in that fashion.
Below is a sample set of the data I am manipulating.
1149621887 1149623161 stormchaser 69.0.0.0 69.0.0.0 +NG004193@net.NET 3 820 156 17570249 0 0 0 + 0 0 0 176 1149621900 1149623197 stormchaser 69.0.0.0 69.0.0.0 +NG004193@net.NET 1 648 80 00 0 0 0 + 0 0 93 1149622376 1149624912 stormchaser 69.0.0.0 69.0.0.0 +NG004193@net.NET 3 7932259 243 85227944 0 0 0 + 0 0 0 268 1149625677 1149625693 stormchaser 213.0.0.0 213.0.0.0 + Sinkbad@net.net 1 49385 0 00 0 0 0 + 0 0 1
Now in the above both the first and second line elements are unixtime stamps the first the start time for the query and the second the stop time for the query. I want to take the second and convert that to the actual date and store the total sum for that date all together (rather make a single DB insert for the given date and if there are two date two inserts one with the first dates total data and the second with the other dates total data). I have no idea of where to start to get where I need to be so I am here asking for the help of my fellow monks. If anyone has any suggestions on this matter please share with a lost monk.
my $sum =0; my $firststart; my $laststop; #my $unixtime; foreach my $line(@dataSUM) { my @lineData = split(/\t/, $line); my $startime = $lineData[0] if(&is_numeric($lineData[0])) || + die "$lineData[0] not numeric\n"; my $stoptime = $lineData[1] if(&is_numeric($lineData[1])) || + die "$lineData[1] not numeric\n"; # $unixtime = scalar localtime $lineData[1] if(&is_numeric( +$lineData[1])) || die "$lineData[1] not numeric\n"; my $xover_bytes = $lineData[7] if(&is_numeric($lineData[7])) || + die "$lineData[7] not numeric\n"; my $art_bytes = $lineData[9] if(&is_numeric($lineData[9])) || + die "$lineData[9] not numeric\n"; my $list_bytes = $lineData[11] if(&is_numeric($lineData[11])) || + die "$lineData[11] not numeric\n"; my $newnews_bytes = $lineData[13] if(&is_numeric($lineData[13])) || + die "$lineData[13] not numeric\n"; my $lineSum = $xover_bytes + $art_bytes + $list_bytes + $newn +ews_bytes; $sum += $lineSum; $firststart = ($startime < $firststart) || $firststart ==0 ? $star +time : $firststart; $laststop = ($stoptime > $laststop ) ? $stoptime : $laststop; } if($DEBUG) { print "This is our sum $sum\n our start $start\n our stop $stop\n"; } sub is_numeric { my $data = shift; return($data =~ m/^\d+$/); } sleep $MySQLrandom; openlog('statsdetailed', 'pid', 'user'); my $usagedbhconnect = DBI->connect("DBI:mysql:host=$dbhost2;database=$ +usagedb", $usagedbuser, $usagedbpass) || die syslog('alert', '%s', "MySQL con +nect to $dbhost2 failed data_source: $DBI::errstr") && print LOG "Connection to $usagedb at +$dbhost2 failed at $extension\n" if($LOG); if($usagedbhconnect && $LOG) { print LOG "Connection to $usagedb was a success\n"; } my $usagesqlQuery = "INSERT INTO usageSum (id,host,reseller,dat +e,startTime,StopTime,totalSum) values (?,?,?,?,?,?,?)"; my $usagesthloc = $usagedbhconnect->prepare($usagesqlQuery); $usagesthloc->execute("NULL","$usageHost","gl_design","NULL","$firstst +art","$laststop","$sum") || die syslog('alert', '%s', "MySQL execut +e failed command: $DBI::errstr") && print LOG "usage DB insertion failed at +$extension VALUES(NULL,$usageHost, \"gl_design\",$firststart,$la +ststop,$sum)\n" if($LOG); if($usagesthloc && $LOG) { print LOG "usage DB insertion was a success\n"; } $usagesthloc->finish; $usagedbhconnect->disconnect;
SUNADMN
USE PERL