Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Cron update of AWStats and database for "country report" feature.

by Alexander (Sexton)
on Nov 04, 2004 at 10:41 UTC ( #405146=sourcecode: print w/replies, xml ) Need Help??
Category: Web Stuff
Author/Contact Info Alexander
Description: Update AWStats and keep a current database for the country reporting feature. Point cron to update_awstats.pl and set the directory variables to match your configuration.
    Three scripts:
  • update_awstats.pl Perl control script, and fetches gzipped country database updates
  • update_geo-ip_db.sh Shell script to gunzip the compressed update and replace the old database file.
  • update_awstats.sh Shell script to update AWStats.
AWStats requires Geo::IP or Geo::IP::PurePerl to use the country reporting feature. The modules in turn are dependant on a database file called GeoIP.dat which is updated about every month.

Both versions of Geo::IP assume users have the correct file permissions to /usr/local/share, and hard code the path /usr/local/share/GeoIP/GeoIP.dat for the database directly into the module.

I do not have the needed file permissions on my server to the /usr/local/share area which created a problem.

I wrote my scripts with a correction to this problem in mind. Directory paths are in variables at the top of each script.

Rather than changing the large AWStats program, I opted for changing the directory paths within the module before installing it. If you have similar file permission problems, this may work for you also.

I have successfully run this on my server which uses FreeBSD 5.2.1, Apache 1.3.26, and Perl 5.8.2.

My directory structure used is:
HOME=/home/username
WWWROOT=/home/username/www
CRONSCRIPTS=/home/username/protected/cron/awstats
ALT_DB_DIR=/home/username/protected/db

NOTE (Nov. 7, 2004): I found a minor glitch I had not noticed in update_awstats.pl earlier. I have changed the code provided below to fix it.

Original line:
if (mirror("$db_website$db_gz_file", "$db_gz_file") != RC_NOT_MODIFIED) {

Corrected line:
if (mirror("$db_website$db_gz_file", "$script_dir/$db_gz_file") != RC_NOT_MODIFIED) {

When executing the script from outside the $script_dir, the mirrored file would instead be placed in the pwd from which the script was called. The corrected line now prevents this from happening.

Here are the scripts:

update_awstats.pl
#!/usr/bin/perl -w
# perl error reporting switches
#    -c = check the code for syntax errors
#    -w = "use warnings" and tells the interpreter to produce useful d
+iagnostics
#    -T = "taint mode" which adds security internal to Perl
#

use strict;

######################################################

my $script_dir = "$ENV{'HOME'}/protected/cron/awstats";
my $db_website = "http://www.maxmind.com/download/geoip/database/";
my $db_gz_file = "GeoIP.dat.gz";

######################################################

my $logfile = "$script_dir/GeoIP.dat.gz.log";



######################################################

my $sh_gz_file = "update_geo-ip_db.sh";
my $sh_awstats = "update_awstats.sh";


use LWP::Simple;

if (mirror("$db_website$db_gz_file", "$script_dir/$db_gz_file") != RC_
+NOT_MODIFIED) {
    # DB has been updated. Log the time and date of the update.
    &Write_Log;
    
    # Run the update_geo-ip_db.sh script to extract and replace the DB
+ file.
    chdir "$script_dir";
    system("./$sh_gz_file");
    
} else {
    # DB already current, just update awstats
    chdir "$script_dir";
    system("./$sh_awstats");
    
}

sub Write_Log {
    use Fcntl qw(:DEFAULT :flock :seek :mode);                # import
+ keywords for
                                                            # flock, s
+eek, and
                                                            # chmod (w
+hile strict
                                                            # subs is 
+in use)
    my $time = gmtime();
    my $file;
    open($file, "+< $logfile")
        or die "Can't open file: $!";
    flock($file, LOCK_EX)
        or die "Can't establish file lock on file: $!";

    my $checkfile = <$file>;
    chomp $checkfile;
    die "I am missing the log file!!!!" unless $checkfile;
    
    print $file "$time GMT\n";

    close ($file);
}

exit 0;

update_geo-ip_db.sh
#!/bin/sh
####################################################
# $HOME/protected/cron/awstats/update_geo-ip_db.sh #
####################################################
# Extract and replace the GeoIP.dat DB file.

# Set variables
DB_DIR=$HOME/protected/db    # Database directory
DB_GZ=GeoIP.dat.gz            # Gzipped DB file

DB_PL=update_awstats.pl        # DB Perl update script

####################################################
# Script, no edit below this point.

cp $DB_GZ $DB_DIR/$DB_GZ
gunzip -f $DB_DIR/$DB_GZ    # -f (force overwrite)
perl $DB_PL

exit 0

update_awstats.sh
#!/bin/sh
##################################################
# $HOME/protected/cron/awstats/update_awstats.sh #
##################################################
# Update AWStats.

# Set variables
AWSTATS_DIR=$HOME/www/path/to/awstats/                        # AWStat
+s directory
AWSTATS_CMD="awstats.pl -config=www_yousite_com -update >/dev/null"   
+ # AWStats update command line

####################################################
# Script, no edit below this point.

perl $AWSTATS_DIR$AWSTATS_CMD

exit 0

I figure there are other monks who use AWStats and may find these scripts useful. I hope you will.

-Alexander
Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: sourcecode [id://405146]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (5)
As of 2019-12-07 16:26 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Strict and warnings: which comes first?



    Results (162 votes). Check out past polls.

    Notices?