Log Rotation

[Danikar]

Category:

Utility Scripts

Description:

A small script I wrote for work. My first script that is going to be used for anything real, so I am pretty nervous putting it into practice tomorrow.

I figured I would throw it up here and see if anyone had any suggestions and so forth.

The key thing is that so far it seems to work. Also, I know very little about log rotation, so hopefully I got the concept down heh.

I used a few `command` to archive old logs, I am sure there is a perl module out there for it, but time was for the essence in here. Any suggestions to switch that stuff out with I am open to.

Date time stamp isn't working correctly. =\

#/usr/bin/perl

use strict;

my $dir = "/var/log";            # Location of the Log file
my $logfile = "log.txt";        # Name of the Log file
my $maxsize = 2_000_000;        # Max size before rotating
my $maxlogs = 10;            # Number of rotates before archive
my $archivetotal = 5;            # Number of logs to archive
my ($s, $m, $h, $dayOfMonth, $month,                # Create date from
+ local time
    $yearOffset, $dow, $doy, $dst) = localtime();
my $year = 1900 + $yearOffset;
my $date = $year * 1000 + ($month + 1) * 100 + $dayOfMonth;    # Forma
+t Date
my $tarme;    # Store names of files to be archived

### Switch to directory where log file is located
chdir $dir;

### Check Size of log
if(-s $logfile > $maxsize)
{
    ### Rotate old logs
    for(my $i = $maxlogs - 1;$i > 0;$i--)
    {
        if(-e "${logfile}.$i")
        {
            my $j = $i + 1;
            rename "${logfile}.$i", "${logfile}.$j";
        }
    }
    rename "$logfile", "${logfile}.1";
    `echo "" >> $logfile`;
    
    ### Archive logs
    if(-e "${logfile}.$maxlogs")
    {
        for(my $i = $maxlogs - ($archivetotal - 1), my $j = 1;$i <= $m
+axlogs;$i++, $j++)
        {
            rename "${logfile}.$i", "${logfile}.${date}.$j";
            $tarme .= "${logfile}.${date}.$j ";
        }
        
        my $i = 1;
        my $tarfile = "${logfile}.${date}.tar";
        while ( (-e $tarfile) || (-e "${tarfile}.gz") )
        {
            $tarfile = "${logfile}.${date}.tar.$i";
            $i++;
        }
        
        `tar -czf $tarfile $tarme`;
        `rm $tarme`;
        `gzip $tarfile`;
    }
}
else
{
    print "Log file too small to rotate.\n\n";
}

[andreas1234567]

I am sure there is a perl module out there for it

Indeed, in fact there are several, e.g. Log::Dispatch::FileRotate and Logfile::Rotate.

but time was for the essence in here.

If I were you I would definitely use a module from CPAN rather than roll my own. Time and effort saved, and the resulting code would probably be better tested.

[Danikar]

Thanks, i'll look at those.

[tirwhan]

One gotcha you need to be aware of, (on *NIX at least), if your application/daemon still has the logfile open for writing your rename in line 28/31 will not remove that association. Instead, the application will continue writing to the logfile.1 and will do so until you eventually remove it, so you might end up losing data between the tar and the rm (consider using the Perl builtin unlink here instead BTW). Also, your application might get confused if the file it is writing to (and for which it holds a valid open file descriptor) suddenly vanishes underneath it.

One method of dealing with this is to send a HUP signal to the application (provided you know which process it is and can figure out the process id) after renaming the file. Most long-running daemons take that as a signal to reopen log files and will do the right thing. You could also figure out which processes have an open file descriptor on the log file (by using the *NIX utility lsof for example) and then send a SIGHUP to that process. Or use one of the modules andreas1234567 pointed out, they have methods for dealing with this exact problem.

Lastly, a non-perl method: take a look at logrotate, it's a common utility used for rotating logs with all bells and whistles built in.

[Danikar]

Thanks, I will probably go ahead and use logrotate for this particular situation. Though, I'll still continue on my perl script for fun =).

[tcf03]

I will sometimes try to reinvent the wheel. Here is my effort. I wrote this several years ago, before I learned to love CPAN and stop reinventing the wheel. I added the syslog piece recently when I was testing Sys::Syslog for inclusion in other programs I am working on. A disclaimer - I do not use this in production.

package MyMods::Log::Logroll;

use strict;
use warnings;
use Sys::Syslog;
use Carp;

our $VERSION = '0.01';

# Syslogging...
my $slog = \&do_syslog;

my %config = ();

sub new
{
    %config = @_;
    confess "Baselog not defined" unless $config{'baselog'};

    $config{'maxbytes'} = 102400 unless $config{'maxbytes'};
    $config{'maxlogs'}  = 5 unless $config{'maxlogs'};
    $config{'UID'} =  ( exists $config{'strict'} == 1 ) ? $< : $>;

    #return ( &rollit() == 0 ) ? 0 : 1;
}

sub roll
{
    my $logowner;
    my $filebytes;

    my $last = $config{'maxlogs'}; 

    if ( -f $config{'baselog'} and ! -x $config{'baselog'} )
    {
        #  0 dev      device number of filesystem
        #  1 ino      inode number
        #  2 mode     file mode  (type and permissions)
        #  3 nlink    number of (hard) links to the file
        #  4 uid      numeric user ID of file's owner
        #  5 gid      numeric group ID of file's owner
        #  6 rdev     the device identifier (special files only)
        #  7 size     total size of file, in bytes
        #  8 atime    last access time in seconds since the epoch
        #  9 mtime    last modify time in seconds since the epoch
        # 10 ctime    inode change time in seconds since the epoch (*)
        # 11 blksize  preferred block size for file system I/O
        # 12 blocks   actual number of blocks allocated
        my @info = stat($config{'baselog'});
        $logowner = $info[4];
        $filebytes = $info[7];
    }
    else
    {
        $slog->("$config{'baselog'} is NOT a regular file");
        return 1;
    }
    
    # File has not reached $maxbytes yet.
    return 0 if $config{'maxbytes'} > $filebytes;
    
    # obviously root can do anything
    if ( ! $config{'UID'} == $logowner and ! $config{'UID'} == 0 )
    {
        $slog->("uid $config{'UID'} is trying to logroll $config{'base
+log'} (owner is $logowner)");
        return 1;
    }    
    
    while ( $last > 1 )
    {
       $last--;
       $slog->("renaming $config{'baselog'}");
       rename_logs($last);
    }
    
    if ( -e $config{'baselog'} ) 
    {
        rename( "$config{'baselog'}", "${config{'baselog'}}_1" ); 
        $slog->("Error renaming $config{'baselog'} -> ${config{'baselo
+g'}}_1")
            unless -e "${config{'baselog'}}_1";
    }

    return ( touchfile() == 0 ) ? 0 : 1;
}    

sub touchfile
{
    my $BASELOG;
    unless ( open $BASELOG, ">$config{'baselog'}" )
    {
         $slog->("Unable to open $config{'baselog'}");
         return 1;
    }
    return ( close $BASELOG ) ? 0 : 1;
}

sub rename_logs
{
     my $x = shift;
     my $y = $x + 1;

     if ( -e "${config{'baselog'}}_$x" )
     {
        $slog->("renaming ${config{'baselog'}}_$x -> ${config{'baselog
+'}}_$y"); 

        rename("${config{'baselog'}}_$x", "${config{'baselog'}}_$y");

        my $return = ( -e "${config{'baselog'}}_$y" ) ? 0 : 1;

        $slog->("Error renaming ${config{'baselog'}}_$x -> ${config{'b
+aselog'}}_$y")
            unless $return == 0;

        return $return;
     }
}

sub do_syslog
{
    my $message = shift;

    openlog( $0, "ndelay,pid", "local0");
    syslog("info", $message);
    closelog;
}

1;