Resizing MRTG (RRDTool) logs en-masse

[McDarren]

MRTG is a very widely-used and popular tool for graphing data. Most typically, it's used for graphing bandwidth utilisation, but it can be (and is) used to graph just about anything.

When used together with RRDTool, MRTG will by default create rrdfiles giving approximately 2 days of 5 minute data, 1 week of 30 minute data, 2 months of 2 hour data & 2 years of one day data.

At my $workplace, we decided that we needed to change these defaults, and fortunately recent versions of MRTG provide a suite of RRDRowCount configuration options for this purpose. And RRDTool provides a resize command for resizing existing rrd's.

My issue was that we had more than 2000 existing rrd's, and some of them had already been resized at an earlier stage. So I needed to iterate through the lot and examine them one by one, and resize accordingly.

Perl to the rescue :-)
The below script ran through the whole lot in just a few minutes.

Disclaimer: If you decide to use the below on your own rrd's, I strongly recommend a dry run on a copy of your files first. You have been warned :-p

Update: Made a couple of small changes as per comments from jwkrahn.

Update 2012-01-29: Now on GitHub.

Cheers,
Darren

#!/usr/bin/perl
use strict;
use warnings;
use RRDs;
use Time::HiRes qw/time/;

my $rrdtool = '/usr/bin/rrdtool';
my $logsdir = 'logs';  # Where all the rrd files live

# The number of data sources in each rrd file
# Typically, for mtrg-generated rrds this will be 8
my $datasources = 8;

my %wanted = (
    1   => 8640,    # 30 days of 5 minute data
    6   => 17520,   # 365 days of 30 min data
    24  => 13140,   # 3 years of 2 hour data
    288 => 3650,    # 10 years of 1 day data
    );

opendir(DIR, $logsdir) or die "Cannot open $logsdir:$!\n";
my @rrds = grep { /.rrd$/ && -f "$logsdir/$_" } readdir DIR;
closedir DIR;
my $numfiles = scalar @rrds;
print "Starting, found $numfiles rrd files\n\n";
my $start = time;

for my $rrd (sort @rrds) {
    print "\nProcessing $rrd\n";
    my $info = RRDs::info "$logsdir/$rrd";
    # Check to ensure we actually have a valid rrd file
    unless ($info->{filename}) {
        print qq|"$logsdir/$rrd" doesn't appear to be a valid rrd log,
+ skipping\n|;
        next;
    }
    for (0 .. $datasources -1) {
        my $cmd = qq|$rrdtool resize $logsdir/$rrd |;
        my $pdp = $info->{"rra[$_].pdp_per_row"};
        my $rows = $info->{"rra[$_].rows"};
        my $cf = $info->{"rra[$_].cf"};
        my $diff = $rows - $wanted{$pdp};
        printf("\tCurrent DS => PDP per row:%.f Rows:%.f CF:%s\n", $pd
+p, $rows, $cf); 
        if ($diff < 0) {
            $diff = abs($diff);
            $cmd .= qq|$_ GROW $diff|;
        }
        elsif ($diff > 0) {
            $cmd .= qq|$_ SHRINK $diff|;
        }
        else {
            print "\tNo change to this DS\n\n";
            next;
        }

        print "\tResizing to $wanted{$pdp} rows, executing $cmd\n";
        system($cmd) == 0 or die "Could not execute $cmd:$!\n";
        print "\tRenaming resized file\n";
        rename 'resize.rrd', "$logsdir/$rrd";
        print "\tDone.\n";
    }
}

my $end = time;
my $dur = sprintf("%.2f", $end - $start);
print "Finished, processed $numfiles files in $dur seconds\n\n";