Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

Log Rotation

by Danikar (Novice)
on Nov 23, 2007 at 08:15 UTC ( [id://652522]=sourcecode: print w/replies, xml ) Need Help??
Category: Utility Scripts
Author/Contact Info danikar@gmail.com
Description:

A small script I wrote for work. My first script that is going to be used for anything real, so I am pretty nervous putting it into practice tomorrow.

I figured I would throw it up here and see if anyone had any suggestions and so forth.

The key thing is that so far it seems to work. Also, I know very little about log rotation, so hopefully I got the concept down heh.

I used a few `command` to archive old logs, I am sure there is a perl module out there for it, but time was for the essence in here. Any suggestions to switch that stuff out with I am open to.

Date time stamp isn't working correctly. =\

#/usr/bin/perl

use strict;

my $dir = "/var/log";            # Location of the Log file
my $logfile = "log.txt";        # Name of the Log file
my $maxsize = 2_000_000;        # Max size before rotating
my $maxlogs = 10;            # Number of rotates before archive
my $archivetotal = 5;            # Number of logs to archive
my ($s, $m, $h, $dayOfMonth, $month,                # Create date from
+ local time
    $yearOffset, $dow, $doy, $dst) = localtime();
my $year = 1900 + $yearOffset;
my $date = $year * 1000 + ($month + 1) * 100 + $dayOfMonth;    # Forma
+t Date
my $tarme;    # Store names of files to be archived

### Switch to directory where log file is located
chdir $dir;

### Check Size of log
if(-s $logfile > $maxsize)
{
    ### Rotate old logs
    for(my $i = $maxlogs - 1;$i > 0;$i--)
    {
        if(-e "${logfile}.$i")
        {
            my $j = $i + 1;
            rename "${logfile}.$i", "${logfile}.$j";
        }
    }
    rename "$logfile", "${logfile}.1";
    `echo "" >> $logfile`;
    
    ### Archive logs
    if(-e "${logfile}.$maxlogs")
    {
        for(my $i = $maxlogs - ($archivetotal - 1), my $j = 1;$i <= $m
+axlogs;$i++, $j++)
        {
            rename "${logfile}.$i", "${logfile}.${date}.$j";
            $tarme .= "${logfile}.${date}.$j ";
        }
        
        my $i = 1;
        my $tarfile = "${logfile}.${date}.tar";
        while ( (-e $tarfile) || (-e "${tarfile}.gz") )
        {
            $tarfile = "${logfile}.${date}.tar.$i";
            $i++;
        }
        
        `tar -czf $tarfile $tarme`;
        `rm $tarme`;
        `gzip $tarfile`;
    }
}
else
{
    print "Log file too small to rotate.\n\n";
}
Replies are listed 'Best First'.
Re: Log Rotation
by andreas1234567 (Vicar) on Nov 23, 2007 at 09:06 UTC
    I am sure there is a perl module out there for it
    Indeed, in fact there are several, e.g. Log::Dispatch::FileRotate and Logfile::Rotate.
    but time was for the essence in here.
    If I were you I would definitely use a module from CPAN rather than roll my own. Time and effort saved, and the resulting code would probably be better tested.
    --
    Andreas
      Thanks, i'll look at those.
Re: Log Rotation
by tirwhan (Abbot) on Nov 23, 2007 at 12:36 UTC

    One gotcha you need to be aware of, (on *NIX at least), if your application/daemon still has the logfile open for writing your rename in line 28/31 will not remove that association. Instead, the application will continue writing to the logfile.1 and will do so until you eventually remove it, so you might end up losing data between the tar and the rm (consider using the Perl builtin unlink here instead BTW). Also, your application might get confused if the file it is writing to (and for which it holds a valid open file descriptor) suddenly vanishes underneath it.

    One method of dealing with this is to send a HUP signal to the application (provided you know which process it is and can figure out the process id) after renaming the file. Most long-running daemons take that as a signal to reopen log files and will do the right thing. You could also figure out which processes have an open file descriptor on the log file (by using the *NIX utility lsof for example) and then send a SIGHUP to that process. Or use one of the modules andreas1234567 pointed out, they have methods for dealing with this exact problem.

    Lastly, a non-perl method: take a look at logrotate, it's a common utility used for rotating logs with all bells and whistles built in.


    All dogma is stupid.
      Thanks, I will probably go ahead and use logrotate for this particular situation. Though, I'll still continue on my perl script for fun =).
        I will sometimes try to reinvent the wheel. Here is my effort. I wrote this several years ago, before I learned to love CPAN and stop reinventing the wheel. I added the syslog piece recently when I was testing Sys::Syslog for inclusion in other programs I am working on. A disclaimer - I do not use this in production.
        package MyMods::Log::Logroll; use strict; use warnings; use Sys::Syslog; use Carp; our $VERSION = '0.01'; # Syslogging... my $slog = \&do_syslog; my %config = (); sub new { %config = @_; confess "Baselog not defined" unless $config{'baselog'}; $config{'maxbytes'} = 102400 unless $config{'maxbytes'}; $config{'maxlogs'} = 5 unless $config{'maxlogs'}; $config{'UID'} = ( exists $config{'strict'} == 1 ) ? $< : $>; #return ( &rollit() == 0 ) ? 0 : 1; } sub roll { my $logowner; my $filebytes; my $last = $config{'maxlogs'}; if ( -f $config{'baselog'} and ! -x $config{'baselog'} ) { # 0 dev device number of filesystem # 1 ino inode number # 2 mode file mode (type and permissions) # 3 nlink number of (hard) links to the file # 4 uid numeric user ID of file's owner # 5 gid numeric group ID of file's owner # 6 rdev the device identifier (special files only) # 7 size total size of file, in bytes # 8 atime last access time in seconds since the epoch # 9 mtime last modify time in seconds since the epoch # 10 ctime inode change time in seconds since the epoch (*) # 11 blksize preferred block size for file system I/O # 12 blocks actual number of blocks allocated my @info = stat($config{'baselog'}); $logowner = $info[4]; $filebytes = $info[7]; } else { $slog->("$config{'baselog'} is NOT a regular file"); return 1; } # File has not reached $maxbytes yet. return 0 if $config{'maxbytes'} > $filebytes; # obviously root can do anything if ( ! $config{'UID'} == $logowner and ! $config{'UID'} == 0 ) { $slog->("uid $config{'UID'} is trying to logroll $config{'base +log'} (owner is $logowner)"); return 1; } while ( $last > 1 ) { $last--; $slog->("renaming $config{'baselog'}"); rename_logs($last); } if ( -e $config{'baselog'} ) { rename( "$config{'baselog'}", "${config{'baselog'}}_1" ); $slog->("Error renaming $config{'baselog'} -> ${config{'baselo +g'}}_1") unless -e "${config{'baselog'}}_1"; } return ( touchfile() == 0 ) ? 0 : 1; } sub touchfile { my $BASELOG; unless ( open $BASELOG, ">$config{'baselog'}" ) { $slog->("Unable to open $config{'baselog'}"); return 1; } return ( close $BASELOG ) ? 0 : 1; } sub rename_logs { my $x = shift; my $y = $x + 1; if ( -e "${config{'baselog'}}_$x" ) { $slog->("renaming ${config{'baselog'}}_$x -> ${config{'baselog +'}}_$y"); rename("${config{'baselog'}}_$x", "${config{'baselog'}}_$y"); my $return = ( -e "${config{'baselog'}}_$y" ) ? 0 : 1; $slog->("Error renaming ${config{'baselog'}}_$x -> ${config{'b +aselog'}}_$y") unless $return == 0; return $return; } } sub do_syslog { my $message = shift; openlog( $0, "ndelay,pid", "local0"); syslog("info", $message); closelog; } 1;
        Ted
        --
        "That which we persist in doing becomes easier, not that the task itself has become easier, but that our ability to perform it has improved."
          --Ralph Waldo Emerson

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: sourcecode [id://652522]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others musing on the Monastery: (3)
As of 2024-11-10 08:03 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    chatterbot is...






    Results (37 votes). Check out past polls.