Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Backup script

by svetho (Beadle)
on Jun 14, 2009 at 17:11 UTC ( #771444=CUFP: print w/ replies, xml ) Need Help??

I keep forgetting making backups of my dotfiles (and other files and directories too), so I wrote the following script that somewhat automates the process. It can be run by cron.
#!/usr/bin/perl # # Script to backup dot- and other files # # Copyright (C) 2009, Sven-Thorsten Fahrbach # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see <http://www.gnu.org/licenses>. use strict; use warnings; use File::Path; use File::Basename; use File::Spec; use File::Copy; use Cwd; use Getopt::Long; use Fcntl qw/ :DEFAULT :flock /; my $VERSION = "0.9"; ############################################################## # VARIABLE DECLARATIONS AND THE LIKE ############################################################## # get a timestamp in the format YYYYMMDDHMS my ($year, $month, $dayofmonth, $hour, $minute, $second) = (localtime())[5, 4, 3, 2, 1, 0]; $year += 1900; $month++; my $timestamp = "$year$month$dayofmonth-$hour$minute$second"; # standard values for config file/command line options my $dotbackup_home = get_dotbackup_home(); my $working_dir = $ENV{HOME}; my $filename = "dotbackup$timestamp"; my $destination_dir = getcwd; my $path_to_conf_file = get_path_to_conf_file(); my $path_to_log_file = File::Spec->catfile($dotbackup_home, "log"); my $path_to_file_list = File::Spec->catfile($dotbackup_home, "files" +); my $compression_level = 1; my $gzip_options = ''; my $tar_options = ''; my $show_help = ''; my $show_version = ''; # this is the array with the files that are to be backed up my @backup_files = get_files(); # $log_level can be: # 0 - quiet mode, no logging # 1 - errors # 2 - warnings # 3 - info # 4 - debug my $log_level = 4; # Get values from the configuration file. Default values will be overw +ritten # with values from the config. parse_config($path_to_conf_file) if $path_to_conf_file; # Get command line options. These will overwrite default options and v +alues # we got from the configuration file. my $result = GetOptions("working-dir=s" => \$working_dir, "filename=s" => \$filename, "destination-dir=s" => \$destination_dir, "logfile=s" => \$path_to_log_file, "file-list=s" => \$path_to_file_list +, "log-level=i" => \$log_level, "compression-level=i" => \$compression_level +, "gzip-options=s" => \$gzip_options, "tar-options=s" => \$tar_options, "help" => \$show_help, "version" => \$show_version); help_and_exit() if $show_help; version_and_exit() if $show_version; $compression_level = 1 if $compression_level < 1; $compression_level = 9 if $compression_level > 9; # commands that have to be invoked by us my $tar = "tar -cf "; my $gzip = "gzip -$compression_level "; #################################################################### # MAIN #################################################################### # declare LOG file handle my $LOG; sysopen ($LOG, $path_to_log_file, O_WRONLY | O_APPEND | O_CREAT) or die "Can't open $path_to_log_file: $!"; # We'll try and keep a lock on the log file for the duration of our sc +ript flock($LOG, LOCK_EX) or die "Can't get a lock on $path_to_log_f +ile"; logger("invoked script", 3); logger("compression level is $compression_level", 3); # try to append leftover command line arguments to our @backup_files foreach (@ARGV) { my $cur_file; unless (File::Spec->file_name_is_absolute($_)) { $cur_file = File::Spec->catfile($working_dir, $_); } next unless -e $cur_file; push @backup_files, File::Spec->canonpath($cur_file); } if (@backup_files == 0) { print "Nothing to do\n"; logger("file stack empty - nothing to do", 3); exit(0); } # build tar-string $tar .= "$tar_options " if $tar_options; my $absolute_filename = File::Spec->catfile($destination_dir, $filenam +e . ".tar"); $tar .= "$absolute_filename "; foreach (@backup_files) { # Remove trailing slash or else tar will archive to contents of th +e # directory instead of the directory itself which is probably not +what # we want. s/\/$//; $tar .= "$_ "; } logger("Archiving " . scalar(@backup_files) . " entries", 3); logger("tar string: $tar", 4); # invoke tar my $tar_result; $tar_result = `$tar`; if ($!) { logger("tar error: $!", 1); logger("exited abnormally", 1); die "tar exited abnormally: $!"; } logger("tar exited successfully", 4); logger("tar output: $tar_result", 4); # invoke gzip $gzip .= "$gzip_options " if $gzip_options; $gzip .= $absolute_filename; my $gzip_result; $gzip_result = `$gzip`; if ($!) { logger("gzip error: $!", 1); logger("gzip exited abnormally", 1); die "gzip exited abnormally: $!"; } logger("gzip exited successfully", 4); logger("gzip output: $gzip_result", 4); logger("script exiting successfully", 3); # release lock on log file, close FH and exit successfully flock($LOG, LOCK_UN); close($LOG); exit(0); ############################################################### # SUBROUTINES ############################################################### sub parse_config { my $conf_filename = shift; unless (open(CONF, $conf_filename)) { warn "Could not open config $conf_filename"; return; } while (<CONF>) { chomp; next if /^\s*$/; # ignore blank lines next if /^\s*#/; # ignore lines consisting of comments +only if (/^working-dir\w*=\w*(.+)$/) { unless (-d $1) { warn "in $conf_filename line $.: Directory does not ex +ist"; next; } $working_dir = $1; next; } if (/^destination-dir\s*=\s*(.+)$/) { unless (-d $1) { warn "in $conf_filename line $.: Directory does not ex +ist"; next; } $destination_dir = $1; next; } if (/^log-file\s*=\s*(.+)$/) { if (-d $1) { warn "in $conf_filename line $.: $1 is a directory"; next; } unless (-e $1) { unless (open(LOG, ">", $1)) { warn "in $conf_filename line $.: Cannot open $1 fo +r writing: $!"; next; } print LOG "[" . scalar(localtime()) . "] Created logfi +le\n"; } close(LOG); $path_to_log_file = $1; next; } if (/^file-list\s*=\s*(.+)$/) { if (-d $1) { warn "in $conf_filename line $.: $1 is a directory"; next; } unless (-e $1) { warn "in $conf_filename line $.: $1 does not exist"; next; } $path_to_file_list = $1; next; } if (/^tar-options\s*=\s*(.*)$/) { $tar_options = $1; } if (/^gzip-options\s*=\s*(.*)$/) { $gzip_options = $1; } if (/^compression-level\s*=\s*(\d)$/) { $compression_level = $1; } if (/^filename\s*=\s*(.+)$/) { $filename = "$1$timestamp"; next; } } } sub get_files { unless (open(FILES, $path_to_file_list)) { warn "Cannot open $path_to_file_list: $!"; return } my @filelist; while (<FILES>) { chomp; next if /^\s*$/; # ignore blank lines next if /^\s*#/; # ignore lines containing only commen +ts my $cur_file; unless (File::Spec->file_name_is_absolute($_)) { $cur_file = File::Spec->catfile($working_dir, $_); } next unless -e $cur_file; push @filelist, File::Spec->canonpath($cur_file); } close(FILES); return @filelist; } sub logger { my ($message, $level) = @_; return if $level > $log_level; # name log levels my @level_names = qw( NONE ERROR WARNING INFO DEBUG ); # autoflush output for current scope only local $| = 1; # DEBUG #print "\@level_names:\n"; #print "\t$_\n" foreach @level_names; #print "\$level: $level\n"; #print "\$message: $message\n"; print $LOG "[" . scalar(localtime()) . "] " . $level_names[$level] + . ": $message\n"; } sub get_dotbackup_home { my $dotbackup_home_path = File::Spec->catdir($ENV{HOME}, ".dotback +up"); unless (-d $dotbackup_home_path) { mkpath($dotbackup_home_path) or die "Could not create dir $dot +backup_home_path: $!"; } return $dotbackup_home_path; } sub get_path_to_conf_file { my $conf_file_path = File::Spec->catfile($dotbackup_home, "conf"); #GetOptions("config=s" => \$path); #return if $path == undef; unless (-e $conf_file_path) { # warn "File $conf_file_path does not exist.\n"; return; } if (-d $conf_file_path) { warn "$conf_file_path is a directory.\n"; return; } return File::Spec->canonpath($conf_file_path); } sub help_and_exit { print <<EOHELP; Usage: $0 [OPTION]... [FILE]... Make a backup from a list of files. Save backup tarball to a specified + path (the current directory by default). -c, --compression-level 1-9, 1 is fastest, 9 is best -d, --destination-dir path to the output directory, default is \ +$HOME --filename filename of the generated tarball (without + the suffix) --file-list path to the list of files to be backed up -g, --gzip-options additionals options to pass to gzip -h, --help give this help --log-level 0-4, 0 is quiet, 4 is the most verbose --logfile path to the log file -t, --tar-options additional options to pass to tar -v, --version display version and exit -w, --working-dir path to the default directory where the fi +le to be backed up are located, default is \$HOME Report bugs to <joran\@alice-dsl.de>. EOHELP exit(0); } sub version_and_exit { print <<EOVERSION; $0 $VERSION Copyright (C) 2009 Sven-Thorsten Fahrbach This is free software. You may redistribute copies of it under the ter +ms of the GNU General Public License <http://www.gnu.org/licenses/gpl.html>. There is NO WARRANTY, to the extent permitted by law. Written by Sven-Thorsten Fahrbach. EOVERSION exit(0); }
The corresponding config file looks like this:
# config file for dotbackup # Entries here override default options. # Command line arguments override config-file options # working-directory: default is ${HOME} # working-directory = /home/sven # destination-directory: default is pwd destination-dir = /home/sven/backups # log-file: default is ~/.dotbackup/log # log-file = /home/sven/.dotbackup/log # file-list: list of files to be backed up, default is ~/.dotbackup/fi +les # file-list = /home/sven/.dotbackup/files # tar-options: additional options to be passed to tar, default is unde +f # Use the following argument to exclude .tmp files from the archive # tar-options = --exclude=*.tmp # gzip-options: dito gzip # use the following option to suppress output from gzip: # gzip-options = --quiet # compression-level: compression level for gzip to be used, default is + 1 # use the following option for best compression (might be slow) # compression-level = 9 compression-level = 3 # filename: filename to be used for the tarball, default is dotbackup ++ # timestamp, do not include file suffixes, these are added automatical +ly # filename = mybackup
Here's an example for a file list, each file has to be on its own line.
.bashrc bin .fetchmailrc .forward .irssi .msmtprc .mutt .muttrc .procmail .procmailrc .purple .urlview .vim .vimrc

Comment on Backup script
Select or Download Code
Re: Backup script
by jwkrahn (Monsignor) on Jun 14, 2009 at 21:01 UTC
    36 # get a timestamp in the format YYYYMMDDHMS 37 my ($year, $month, $dayofmonth, $hour, $minute, $second) 38 = (localtime())[5, 4, 3, 2, 1, 0];

    Why not just assign the variables in the order that localtime returns them:

    # get a timestamp in the format YYYYMMDDHMS my ( $second, $minute, $hour, $dayofmonth, $month, $year ) = localtime;
    70 parse_config($path_to_conf_file) if $path_to_conf_file; 100 # declare LOG file handle 101 my $LOG; 102 sysopen ($LOG, $path_to_log_file, O_WRONLY | O_APPEND | O_CREA +T) 103 or die "Can't open $path_to_log_file: $!"; 104 # We'll try and keep a lock on the log file for the duration o +f our script 105 flock($LOG, LOCK_EX) or die "Can't get a lock on $path_ +to_log_file"; 201 if (/^log-file\s*=\s*(.+)$/) { 202 if (-d $1) { 203 warn "in $conf_filename line $.: $1 is a direc +tory"; 204 next; 205 } 206 unless (-e $1) { 207 unless (open(LOG, ">", $1)) { 208 warn "in $conf_filename line $.: Cannot op +en $1 for writing: $!"; 209 next; 210 } 211 print LOG "[" . scalar(localtime()) . "] Creat +ed logfile\n"; 212 } 213 close(LOG); 214 $path_to_log_file = $1; 215 next; 216 }

    You call parse_config() on line 70 which opens and prints to and closes the log file and then sets the variable $path_to_log_file which is subsequently used to open the log file again.   Do you really need to open the log file inside the parse_config() function?

    268 sub logger { 269 my ($message, $level) = @_; 270 return if $level > $log_level; 271 272 # name log levels 273 my @level_names = qw( NONE ERROR WARNING INFO DEBUG ); 274 275 # autoflush output for current scope only 276 local $| = 1; 277 278 # DEBUG 279 #print "\@level_names:\n"; 280 #print "\t$_\n" foreach @level_names; 281 #print "\$level: $level\n"; 282 #print "\$message: $message\n"; 283 284 print $LOG "[" . scalar(localtime()) . "] " . $level_names +[$level] . ": $message\n"; 285 }

    You are changing the value of the $| variable which turns on autoflush for the currently selected file handle which by default is STDOUT so it will have no effect on the following print statement.   Since you are opening the log file with sysopen perhaps you should write the log message using syswrite which bypasses buffered IO.

    syswrite $LOG "[" . scalar(localtime()) . "] " . $level_names[$lev +el] . ": $message\n";
    202 if (-d $1) { 203 warn "in $conf_filename line $.: $1 is a direc +tory"; 204 next; 205 } 206 unless (-e $1) { 218 if (-d $1) { 219 warn "in $conf_filename line $.: $1 is a direc +tory"; 220 next; 221 } 222 unless (-e $1) {

    To avoid stating the same file twice you can use the special _ filehandle:

    202 if (-d $1) { 203 warn "in $conf_filename line $.: $1 is a direc +tory"; 204 next; 205 } 206 unless (-e _) { 218 if (-d $1) { 219 warn "in $conf_filename line $.: $1 is a direc +tory"; 220 next; 221 } 222 unless (-e _) {
      Thanks for your comment. Your suggestions all make sense and I wasn't even aware of the _ file handle - thanks! I'll implement your improvements and post the updated script here (which will probably take until next weekend).
Re: Backup script
by heptadecagram (Novice) on Jun 18, 2009 at 19:50 UTC

    I would like to point out that there is great utility in putting your home directory's dot files into version control, especially a distributed VCS like hg, bzr, or git.

    That way, not only do you have a backup, but you have a versioned backup, for when you want to find that shell function you deleted a few months back. Also, it's trivially easy to get set up on a new machine (just check out the repository).

      ++, I keep many of my dotfiles in a git repo and can't recommend it enough. Being able to sit down on a virgin box, git clone, and have all my stuff completely customized (including/especially vim) is the bee's knees. Pushing local customizations out to all the boxes I use without having to think about it is even better.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: CUFP [id://771444]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (8)
As of 2014-12-25 03:10 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (159 votes), past polls