Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

How do I ignore an entire directory using File::Find?

by theillien1 (Acolyte)
on Jun 18, 2014 at 00:35 UTC ( #1090229=perlquestion: print w/ replies, xml ) Need Help??
theillien1 has asked for the wisdom of the Perl Monks concerning the following question:

I have a script which looks for world writable files and writes any that are found to an output file or, if it is listed in a file containing excluded files, ignore it and move on to the next. One request for the script is to have it ignore an entire directory.

This is a problem because we have an application installed that (for some unknown reason) sets every...single...file in all of its directories as world writable. The issue with File::Find::prune is that it will only skip the directory a file is found in. The app in question actually has several sub-directories with varying depths.

For instance:

/opt/app/bin/.../* /opt/app/config/.../.../* /opt/app/log/.../.../.../* /opt/app/tmp/*

What I'd rather do is simply tell the script to skip the entire /opt/app directory rather than evaluate a file, determine if it is supposed to be ignored, then set File::Find::prune. Not only would it vastly simplify things, but it would eliminate the need to place a file from every single sub-directory into the excludes list for at least the one evaluation. Additionally, the base directory (/opt/app) does not have any files. Only sub-directories. I wouldn't be able to simply place the one file in the excludes list and have it be the basis for File::Find::prune.

This is the script as I have it now:

#!/usr/bin/perl use warnings; use strict; use Fcntl ':mode'; use File::Find; no warnings 'File::Find'; no warnings 'uninitialized'; my $dir = "/var/log/tivoli/"; my $mtab = "/etc/mtab"; my $permFile = "world_writable_files.txt"; my $tmpFile = "world_writable_files.tmp"; my $exclude = "/usr/local/etc/world_writable_excludes.txt"; my $mask = S_IWUSR | S_IWGRP | S_IWOTH; my (%excludes, %devNums); my $errHeader; # Compile a list of mountpoints that need to be scanned my @mounts; open MT, "<${mtab}" or die "Cannot open ${mtab}, $!"; # We only want the local mountpoints while (<MT>) { if ($_ =~ /ext[34]/) { chomp; my @line = split; push(@mounts, $line[1]); my @stats = stat($line[1]); $devNums{$stats[0]} = undef; } } close MT; # Build a hash from /usr/local/etc/world_writables_excludes.txt if ((! -e $exclude) || (-z $exclude)) { $errHeader = <<HEADER; !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !! !! !! /usr/local/etc/world_writable_excludes.txt is !! !! is missing or empty. This report includes !! !! every world-writable file including those which !! !! are expected and should be excluded. !! !! !! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! HEADER } else { open XCLD, "<${exclude}" or die "Cannot open ${exclude}, $!\n"; while (<XCLD>) { chomp; $excludes{$_} = 1; } } sub wanted { my @dirStats = stat($File::Find::name); # Is it excluded from the report... return if exists $excludes{$File::Find::name}; # ...in a special directory, ... return if ($File::Find::name =~ /^\bsys\b|\bproc\b|\bdev\b$/); # ...a regular file, ... return unless -f; # ...local, ... return unless (exists $devNums{$dirStats[0]}); # ...and world writable? return unless ($dirStats[2] & $mask) == $mask; # If so, add the file to the list of world writable files print(WWFILE "$File::Find::name\n"); } # Create the output file path if it doesn't already exist. mkdir($dir or die "Cannot execute mkdir on ${dir}, $!") unless (-d $di +r); # Create our filehandle for writing our findings open WWFILE, ">${dir}${tmpFile}" or die "Cannot open ${dir}${tmpFile}, + $!"; print(WWFILE "${errHeader}") if ($errHeader); find(\&wanted, @mounts); close WWFILE; # If no world-writable files have been found ${tmpFile} should be zero +-size; # Delete it so Tivoli won't alert if (-z "${dir}${tmpFile}") { unlink "${dir}${tmpFile}"; } else { rename("${dir}${tmpFile}","${dir}${permFile}") or die "Cannot rename + file ${dir}${tmpFile}, $!"; }

As a suggestion for optimization I was told the wanted sub should look like this:

sub wanted { my @dirStats = stat($File::Find::name); # Is it excluded from the report... if (exists $excludes{$File::Find::name}) { $File::Find::prune=1 if (-d _); return; } # ...in a basic directory, ... if ($File::Find::name =~ /^\bsys\b|\bproc\b|\bdev\b$/) { $File::Find::prune=1 if (-d _); return; } # ... not a regular file, ... return unless -f _; # ...local, ... return if (exists $devNums{$dirStats[0]}); # ...and world writable? my $protection = $dirStats[2]; my $writemask = (S_IWUSR | S_IWGRP | S_IWOTH); return unless $writemask == $protection & $writemask; # If so, add the file to the list of world writable files print(WWFILE "$File::Find::name\n"); }

After messing with this I found that it doesn't do what I want and is where the my notion of adding every single sub-directory comes from.

What should I be doing to ensure the /opt/app directory is never traversed and as soon as it is encountered should be skipped entirely? I imagine it would be a matter of running at least one test to determine if a file's base directory is /opt/app.

Comment on How do I ignore an entire directory using File::Find?
Select or Download Code
Re: How do I ignore an entire directory using File::Find? (find/rule)
by Anonymous Monk on Jun 18, 2014 at 01:19 UTC

    find/rule is much easier to manage, see Re^2: getting picky with File::Find::Rule less typing find/rule , write something like

    use File::Find::Rule qw/ find rule /; my $gonerDirs = rule( directory => name => [ 'baby', 'dead' ], qw/ prune discard/, ); ## use File::Find::Rule::Permissions (); ## ??? not for win32 ## my $keeperFiles = rule( file => permissions => [ '???' ],... ); my $keeperFiles = rule( file => exec => \&worldWriteable ... ); my @files = find( not => $gonerDirs , any => $keeperFiles , in => \@startdirs, );

    Same deal

    my @files = rule()->or( ## !IMPORTANT NOTE discard before prune with rule()/find() rule( name => [ 'baby','dead' ], qw/ directory discard prune /), rule( file => exec => \&worldWriteable,... ) )->in( @startdirs );

    Where you write

    sub worldWritable { my( $shortname, $path, $fullname ) = @_; ## $shortname == $_; ... return 1 if $want; return 0 if not $want; }
      I'm not entirely sure I'm following your example.

        I'm not entirely sure I'm following your example.

        Ok then :) do you have a question?

Re: How do I ignore an entire directory using File::Find?
by roboticus (Canon) on Jun 18, 2014 at 05:05 UTC

    theillien1:

    If you provide a preprocess function in File::Find, you can do it easily. Just remove the directory from the list of strings and it will skip the directory entirely.

    ...roboticus

    When your only tool is a hammer, all problems look like your thumb.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1090229]
Approved by snoopy
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (7)
As of 2014-12-25 22:49 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (163 votes), past polls