Beefy Boxes and Bandwidth Generously Provided by pair Networks RobOMonk
The stupid question is the question not asked
 
PerlMonks  

Adding/Committing Files to CVS

by bassplayer (Monsignor)
on Sep 12, 2002 at 22:21 UTC ( #197374=perlquestion: print w/ replies, xml ) Need Help??
bassplayer has asked for the wisdom of the Perl Monks concerning the following question:

Greetings Exalted Monks,

I have been tasked with adding a huge amount of files to CVS. This large web directory was created by others who did not employ CVS (as they should have). I started adding a few files, before realizing what a tremendous amount of time was involved. Then I decided to write some Perl to make my life easier.

Basically, I now have a program that I run in the main directory, and it is supposed to find all acceptible files (.pl, .cgi, .htm, .html, .shtml) and then add and commit them to the repository. It also adds and commits images using the binary setting of CVS. If it finds a directory, it should recurse through it and add and commit files found inside this directory, etc.

I initially found that within directories, the files were not being added or committed, because CVS was being passed file names without full path names. I am using IO::Dir to get directory information. It is not giving me full path names, just file names.

I am storing the directory names in an array, then adding and committing later because I was getting a Deep recursion on subroutine warning.

I apologize for the crappy code, but I wanted to get it working before I later optimized it (and I'm no Perl genius anyway).

I guess my question is, can I even do this the way that I have set out (IO::Dir, Shell)? Is there another way that someone can suggest? Does CVS offer any automation like this?

This should be a simple bit of code, I know, and I'm embarrassed to post this, but at least I'll learn. Anyway, here's the code:

#! /usr/bin/perl -w use strict; use IO::Dir; use Shell; open (LOG_FILE, ">/home/bassplayer/add_to_cvs_log.txt") || die "Couldn +'t open log file"; my @all_directories; my @directories = get_directories( dir => '.', first => '1' ); while ( @directories ) { my @more_directories = get_directories( dir => $directories[0], first => '0' ); push @directories, @more_directories; push @all_directories, $directories[0]; shift @directories; } foreach ( @all_directories ) { print STDOUT "ALL_DIR: $_\n"; check_files( dir => $_ ); } close LOG_FILE; exit 0; ##################### sub get_directories { ##################### my %arg = @_; my @directories; my $d = new IO::Dir $arg{dir}; if ( defined $d ) { while ( defined( $_ = $d->read ) ) { unless ( $_ eq '..' || ( $_ eq '.' && $arg{first} ne '1' ) + || $_ eq 'CVS' ) {; if ( -d $_ ) { print STDOUT "DIR: $_\n"; push @directories, $_; } } } undef $d; } return @directories; } ################# sub check_files { ################# my %arg = @_; my $d = new IO::Dir $arg{dir}; if ( defined $d ) { while ( defined( $_ = $d->read ) ) { next if $_ eq '..' || $_ eq '.' || $_ eq 'CVS'; if ( -d $_ ) { print LOG_FILE "ADDING DIR: $_\n"; my $cvs_output = cvs( 'add', $_ ); print LOG_FILE "COMMITTING DIR: $_\n"; $cvs_output .= cvs( 'commit', '-m _', $_ ); print LOG_FILE $cvs_output; } if ( ( -B $_ ) && ( $_ =~ m/\.jpg$/ || $_ =~ m/\.jpeg$/ || + $_ =~ m/\.gif$/ ) ) { print LOG_FILE "ADDING FILE: $_\n"; my $cvs_output = cvs( 'add', '-kb', $_ ); print LOG_FILE "COMMITTING FILE: $_\n"; $cvs_output .= cvs( 'commit', '-m _', $_ ); print LOG_FILE $cvs_output; } elsif ( $_ =~ m/\.pl$/ || m/\.cgi$/ || m/\.htm$/ || $_ =~ +m/\.html$/ || $_ =~ m/\.shtml$/ ) { print LOG_FILE "ADDING FILE: $_\n"; my $cvs_output = cvs( 'add', $_ ); print LOG_FILE "COMMITTING FILE: $_\n"; $cvs_output .= cvs( 'commit', '-m _', $_ ); print LOG_FILE $cvs_output; } } undef $d; } }

Your help is, of course, very much appreciated. Optimization suggestions are also welcome, as I do not get code review from anywhere, and am very interested in improving my Perl.

bassplayer

Comment on Adding/Committing Files to CVS
Select or Download Code
Re: Adding/Committing Files to CVS
by rbc (Curate) on Sep 12, 2002 at 22:37 UTC
    You may be able to use cvs's import command if the code is not already in the repository.
Re: Adding/Committing Files to CVS
by mfriedman (Monk) on Sep 12, 2002 at 22:44 UTC
    I would reccomend using File::Find to locate all the files you want to commit. It will impliment the recursion for you and return full pathnames.
Re: Adding/Committing Files to CVS
by jkahn (Friar) on Sep 12, 2002 at 22:45 UTC
    CVS itself offers some features to deal with this. A bit dangerous in this forum, but Perl may not be the Right Thing here.

    Instead, I strongly recommend taking a good hard look at the cvs import subcommand, which is a sort of sophisticated recursive cvs add, and reading the documentation for CVS to see how to use its configuration file to skip files.

    In summary, since CVS already handles this it might be a Bad Idea to write your own recursive add routines.

Re: Adding/Committing Files to CVS
by csotzing (Sexton) on Sep 13, 2002 at 10:50 UTC
    In perl, anytime you need to parse a directory tree, File::Find is a good answer. You have options like changing to the directory you're parsing, or going depth-first. You can also get the file's name, parent directory, path, etc.

    Definitely check out the cvs options first.

    -C
    print(map(lc(chr),split(6,qw/99672682673683684689632658645641610607/)));
Re: Adding/Committing Files to CVS
by TStanley (Canon) on Sep 13, 2002 at 11:10 UTC
    As mentioned above, the File::Find module is a good one to use. And in the September issue of Sys Admin magazine, merlyn shows how well it can be used.

    TStanley
    --------
    It is God's job to forgive Osama Bin Laden. It is our job to arrange the meeting -- General Norman Schwartzkopf
Re: Adding/Committing Files to CVS
by bassplayer (Monsignor) on Oct 03, 2002 at 20:29 UTC
    Well, I fiddled with File::Find for a bit, but I couldn't get it to do what I needed. I ended up using opendir. Here is the code I wound up with, in case it might actually benefit someone:
    #! /usr/bin/perl -w use strict; use IO::Dir; use Shell; open (LOG_FILE, ">/home/bassplayer/add_to_cvs/htdocs_out.txt") || die +"Couldn't open log file"; open (DIR_FILE, ">/home/bassplayer/add_to_cvs/htdocs_dir.txt") || die +"Couldn't open dir file"; my @all_directories; my @directories = ( '.' ); while ( @directories ) { my @more_directories = get_directories( dir => $directories[0], first => '0' ); push @directories, @more_directories; push @all_directories, $directories[0]; shift @directories; } print LOG_FILE "TOTAL NUM DIR: $#all_directories\n"; foreach ( @all_directories ) { check_files( dir => $_ ); } close DIR_FILE; close LOG_FILE; exit 0; ##################### sub get_directories { ##################### my %arg = @_; my @directories; my $path = $arg{dir}; opendir(DIR, $arg{dir}) || die "can't opendir $arg{dir}: $!"; my @output = grep { -d "$arg{dir}/$_" } readdir(DIR); for ( my $i = 0 ; $i <= $#output ; $i++ ) { if ( $output[$i] ne '.' && $output[$i] ne '..' && $output[$i] +!~ m/CVS/ && $arg{dir} !~ m/CVS/ ) { my $directory = "$arg{dir}/$output[$i]"; my $cvs_output; print LOG_FILE "ADDING DIR: $directory\n"; $cvs_output = cvs( 'add', $directory ); print LOG_FILE "COMMITTING DIR: $directory\n"; $cvs_output .= cvs( 'commit', '-m _', $directory ); print LOG_FILE $cvs_output; push @directories, $directory; print DIR_FILE "$directory\n"; } } return @directories; } ################# sub check_files { ################# my %arg = @_; my $d = new IO::Dir $arg{dir}; if ( defined $d ) { while ( defined( $_ = $d->read ) ) { next if $_ eq '..' || $_ eq '.' || $_ =~ m/CVS/; my $file = "$arg{dir}/$_"; if ( -d $_ ) { print LOG_FILE "SKIPPING DIR: $file\n"; } elsif ( $_ =~ m/\.jpg$/ || $_ =~ m/\.jpeg$/ || $_ =~ m/\.gif$/ || $_ =~ m/\.xls$/ || $_ =~ m/\.doc$/ || $_ =~ m/\.pdf$/ || $_ =~ m/\.swf$/ || $_ =~ m/\.cgi$/ ) { my $cvs_output; print LOG_FILE "ADDING BINARY FILE: $file\n"; $cvs_output = cvs( 'add', '-kb', $file ); print LOG_FILE "COMMITTING FILE: $file\n"; $cvs_output .= cvs( 'commit', '-m _', $file ); print LOG_FILE $cvs_output; } elsif ( $_ =~ m/\.htm$/ || $_ =~ m/\.html$/ || $_ =~ m/\.shtml$/ || $_ =~ m/\.js$/ || $_ =~ m/\.css$/ || $_ =~ m/\.pl$/ || $_ =~ m/^\.htaccess$/ || $_ =~ m/^\..*passwd$/ || $_ =~ m/^\.users$/ || $_ =~ m/^\.txt$/ ) { my $cvs_output; print LOG_FILE "ADDING FILE: $file\n"; $cvs_output = cvs( 'add', $file ); print LOG_FILE "COMMITTING FILE: $file\n"; $cvs_output .= cvs( 'commit', '-m _', $file ); print LOG_FILE $cvs_output; } } undef $d; } }


    Thanks to everyone who took the time to help me with this.

    bassplayer

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://197374]
Approved by rbc
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others meditating upon the Monastery: (7)
As of 2014-04-16 05:35 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (414 votes), past polls