perladdict has asked for the wisdom of the Perl Monks concerning the following question:
Hi Monks,
There is a requirement to delete the old log lines from event-logs in production server in order to free the disk space Below is the Unix directory structure for each application to navigate to the event-logs.
From the above log i need delete the log lines below the tracking datelines by keeping the date line as it is in the log. Sorry for the long posting,below is the code i am trying to implement this in perl as well in shell script, but no luck...
Above are all the application branch codes except log,lib.old,log,LMPROD,ques,scripts,test,utils,transfer,lib,test.tar Each of the branches are having different applications event-log,the TMP directory is having all the applications event log for a branch I need to navigate to each branch's TMP directory in order to view the event-logs for example: "cd /rms/799/cool/data/TMP" is the path of event-logs for a branch 799.$ cd /rms $ ls 088 533 788 802 916 lib 267 535 789 822 917 lib.old 268 537 790 834 919 LMPROD 270 600 791 841 AXSDBIMG log 272 6000 792 855 CVSROOT lost+found 276 779 794 872 DBG ques 287 781 795 888 DBI scripts 288 783 796 910 DBT scripts.old 406 784 797 911 GENEOS test.tar 450 785 798 912 home transfer 490 786 799 915 KL utils
Each application's event-logs for a branch is having application specific group names and user name, in order to view the event logs i need to log-in with application specific usernames.In order to gain the disk space i need to delete the old log line i.e. log lines which is tracked in "/2011 and before Below is the sample event-log$ cd /rms/799/cool/data/TMP -rw-rw-r-- 1 coolvas1 coolvas1 4325 Jan 4 15:04 COOL-event-log.L +BOX -rw-rw---- 1 coole3 coole3 297 Jan 25 03:01 COOL-event-log.E +3ASIA -rw-rw-r-- 1 coolbrz coolbrz 20526 Feb 12 05:27 COOL-event-log.B +REEZE -rw-rw---- 1 coolmint coolmint 307297 Feb 12 08:30 COOL-event-log.M +INT
>coolmint started sh-MINTlink on 24/08/2012 at 08:30:09 INBC file found, Business date fetched is: 20120823 Received Parameters: APP: MINT, BRANCH: 799 and RECORDING DATE: 201208 +23 Validating file: CS-TWTPOUTC.20120823.1 CS-TWTPINBC.20120823.1 CS-TWTP +OREF.20120823.1 CS-TWTPIREF.20120823.1 CS-TWTPNAK.20120823.1 Transfering file: CS-TWTPOUTC.20120823.1 CS-TWTPINBC.20120823.1 CS-TWT +POREF.20120823.1 CS-TWTPIREF.20120823.1 CS-TWTPNAK.20120823.1 > started MINT recording for procdate 20120823 on 2012/08/24 at 08:30: +10 Application Level BACKDATE_DAYS NOT set, hence considering Branch leve +l days a CS-TWTPOUTC.20120823.1 19K a CS-TWTPINBC.20120823.1 54K a CS-TWTPOREF.20120823.1 1K a CS-TWTPIREF.20120823.1 1K a CS-TWTPNAK.20120823.1 1K ============================================= Adding page break to the EOF for CS-TWTPOUTC.20120823.1 Adding page break to the EOF for CS-TWTPINBC.20120823.1 Adding page break to the EOF for CS-TWTPOREF.20120823.1 Adding page break to the EOF for CS-TWTPIREF.20120823.1 Adding page break to the EOF for CS-TWTPNAK.20120823.1 usage: sleep time ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~ > coolmint started sh-MINTlink on 25/08/2012 at 08:30:09 INBC file found, Business date fetched is: 20120824 Received Parameters: APP: MINT, BRANCH: 799 and RECORDING DATE: 201208 +24 Validating file: CS-TWTPINBC.20120824.1 CS-TWTPOUTC.20120824.1 CS-TWTP +OREF.20120824.1 CS-TWTPIREF.20120824.1 CS-TWTPNAK.20120824.1 Transfering file: CS-TWTPINBC.20120824.1 CS-TWTPOUTC.20120824.1 CS-TWT +POREF.20120824.1 CS-TWTPIREF.20120824.1 CS-TWTPNAK.20120824.1 > started MINT recording for procdate 20120824 on 2012/08/25 at 08:30: +10 Application Level BACKDATE_DAYS NOT set, hence considering Branch leve +l days a CS-TWTPINBC.20120824.1 11K a CS-TWTPOUTC.20120824.1 53K a CS-TWTPOREF.20120824.1 1K a CS-TWTPIREF.20120824.1 1K a CS-TWTPNAK.20120824.1 1K ============================================= Adding page break to the EOF for CS-TWTPINBC.20120824.1 Adding page break to the EOF for CS-TWTPOUTC.20120824.1 Adding page break to the EOF for CS-TWTPOREF.20120824.1 Adding page break to the EOF for CS-TWTPIREF.20120824.1 Adding page break to the EOF for CS-TWTPNAK.20120824.1 usage: sleep time ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~ > coolmint started sh-MINTlink on 27/08/2012 at 08:30:09 INBC file found, Business date fetched is: 20120825 /rms/transfer/799/mint/*OUTC.20120825*: No such file or directory Received Parameters: APP: MINT, BRANCH: 799 and RECORDING DATE: 201208 +25 Validating file: CS-TWTPINBC.20120825.1 CS-TWTPIREF.20120825.1 CS-TWTP +NAK.20120825.1 Transfering file: CS-TWTPINBC.20120825.1 CS-TWTPIREF.20120825.1 CS-TWT +PNAK.20120825.1 > started MINT recording for procdate 20120825 on 2012/08/27 at 08:30: +09 Application Level BACKDATE_DAYS NOT set, hence considering Branch leve +l days a CS-TWTPINBC.20120825.1 7K a CS-TWTPIREF.20120825.1 1K a CS-TWTPNAK.20120825.1 1K ============================================= Adding page break to the EOF for CS-TWTPINBC.20120825.1 Adding page break to the EOF for CS-TWTPIREF.20120825.1 Adding page break to the EOF for CS-TWTPNAK.20120825.1 usage: sleep time ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~ > coolmint started sh-MINTlink on 28/08/2012 at 08:30:09 INBC file found, Business date fetched is: 20120827 Received Parameters: APP: MINT, BRANCH: 799 and RECORDING DATE: 201208 +27 Validating file: CS-TWTPOUTC.20120827.1 CS-TWTPINBC.20120827.1 CS-TWTP +OREF.20120827.1 CS-TWTPIREF.20120827.1 CS-TWTPNAK.20120827.1 Transfering file: CS-TWTPOUTC.20120827.1 CS-TWTPINBC.20120827.1 CS-TWT +POREF.20120827.1 CS-TWTPIREF.20120827.1 CS-TWTPNAK.20120827.1 > started MINT recording for procdate 20120827 on 2012/08/28 at 08:30: +09 Application Level BACKDATE_DAYS NOT set, hence considering Branch leve +l days a CS-TWTPOUTC.20120827.1 14K a CS-TWTPINBC.20120827.1 47K a CS-TWTPOREF.20120827.1 1K a CS-TWTPIREF.20120827.1 1K a CS-TWTPNAK.20120827.1 1K Adding page break to the EOF for CS-TWTPOUTC.20120827.1 Adding page break to the EOF for CS-TWTPINBC.20120827.1 Adding page break to the EOF for CS-TWTPOREF.20120827.1 Adding page break to the EOF for CS-TWTPIREF.20120827.1 Adding page break to the EOF for CS-TWTPNAK.20120827.1 ============================================= usage: sleep time ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~ > coolmint started sh-MINTlink on 29/08/2012 at 08:30:08 INBC file found, Business date fetched is: 20120828 Received Parameters: APP: MINT, BRANCH: 799 and RECORDING DATE: 201208 +28 Validating file: CS-TWTPOUTC.20120828.1 CS-TWTPINBC.20120828.1 CS-TWTP +OREF.20120828.1 CS-TWTPIREF.20120828.1 CS-TWTPNAK.20120828.1 Transfering file: CS-TWTPOUTC.20120828.1 CS-TWTPINBC.20120828.1 CS-TWT +POREF.20120828.1 CS-TWTPIREF.20120828.1 CS-TWTPNAK.20120828.1 > started MINT recording for procdate 20120828 on 2012/08/29 at 08:30: +09
#!/usr/bin/perl -w @branches = `ls -ld /rms/???/cool/data/HOLDING | awk -F" " '{print $NF +}' | cut -f3 -d"/" | sort | uniq`; #print "@branches\n"; foreach(@branches) { chomp $_; $var1 = "/rms/"; $var2 = $_; $var3 = "/cool/data/TMP"; @absdirpath = $var1.$var2.$var3; #print "@absdirpath\n"; foreach(@absdirpath) { @eventlogs = `ls $_ |grep "COOL*`; #print "@eventlogs"; foreach (@eventlogs) { @listing = `ls -lrt $_`; print "@listing"; } #@grp = `ls -lrt | cut -d " " -f6`; #print "@grp\n"; foreach $logs (@eventlogs) { $logs =~ s/^\s//; print "$logs\n"; ($logname,$appname) = split(/\./,$logs); @apps = $appname; #print "@apps"; } } }
Dear monks, please assist me to get done this, as there is a deadline for me in order to avoid outage.Thanks in-advance.# To get the branches for branch in `ls -ld /rms/???/cool/data/HOLDING | awk -F" " '{print $ +NF}' | cut -f3 -d"/" | sort | uniq` do # every branch cd /rms/$branch/cool/data/TMP # To get the respective application for app in `ls -ld COOL-event-log.* | awk -F" " '{prin +t $NF}'|cut -f2 -d"."|sort|uniq` do echo "Performing Deletion of lines before 2011 + for: Application : $app, Branch: $branch" echo "======================================== +======================================== " # To get the respective application group for grp in `ls -ltr COOL-event-log.$app|awk '{print $3}'` do newgrp $grp if [ -f COOL-event-log.$app ] then tot_cnt=`wc COOL-event-log.$app|awk ' { print $1 }'` del_cnt=`grep -n '/2011' COOL-event-log.$app|head -1|cut - +f1 -d":"` rem_cnt=`expr $tot_cnt - $del_cnt - 1` tail -$rem_cnt COOL-event-log.$app > COOL-event-log-new.$a +pp rm -f COOL-event-log.$app mv COOL-event-log-new.$app COOL-event-log.$app sudo chown $grp:$grp COOL-event-log.$app else echo no file for interface $app fi done done done
Back to
Seekers of Perl Wisdom