Descending through directories

[Corion]

In your code, you donīt need to re-read the directory to get at the files in the directory - although it would be difficult to "compute" the difference of the array elegantly (well, difficult to me at least). I mostly use the following way, intermixing files and directories :

  # Untested code - use at your own risk
  sub handledirectory {
    my ($directory) = @_;
    my ($entry, @direntries);
    opendir( DIR, $directory ) or die "Canīt read $directory : $!\n";
    @direntries = readdir( DIR ) or die "Error reading $directory : $!
+\n";
    closedir DIR;

    foreach $entry, @direntries {
      # File::Spec gives us cross-platform path utilities
      # and comes with every Perl standard distribution
      require File::Spec;
      my $fullpath;
      
      # skip current and parent directory entries
      next if $entry =~ /^\.\.?$/;

      $fullpath = File::Spec->catfile( $directory, $entry );
      if (-d $fullpath ) {
        &handledirectory($fullpath);
      } elsif ( -f $fullpath ) {
        # This second call to stat() (implicit in the "-f")
        # could be done away by using some other short
        # variable that does caching, but that would maybe
        # confuse the readers ...
        ... do stuff ...
      } else {
        # something strange ...
      };
  }

[t0mas]

The code I posted was to demonstrate a way to recursive call self, not being very useful in itself... ;-)
There is more than one way to do it. You could rewinddir and readdir again (with a different grep) on the same handle too..
But I agree that your solution is beautiful.
Maybe someone could benchmark some testcases.

/brother t0mas

[Corion]

I've never used perlfunc:rewinddir() :), and my post wasn't meant as an offence, sorry if I came across that way ...

[t0mas]

Did some benchmarking today. I really like knowing the most effective way to solve a certain problem and when Corion posted his code above, I got curious. To Corion, I would like to say that this is no "I'm right - you're wrong" kind of thing. I've enjoyed your code (since I love and use eConsole) for a long time, and I really didn't know which of the ways that was most effective, so please don't take this the wrong way.
If someone else have ideas about this please, give it a shot with your own code.
I use directory travering quite often so I would really be glad to be able to use the most effective code in my programs.
Here we go:


use Benchmark;
use File::Spec;
use File::Find;

$t0 = new Benchmark;
&t1('C:\\Program');
$t1 = new Benchmark;
&t2('C:\\Program');
$t2 = new Benchmark;
&t3('C:\\Program');
$t3 = new Benchmark;
&t4('C:\\Program');
$t4 = new Benchmark;
&t5('C:\\Program');
$t5 = new Benchmark;

print "t1: ",timestr(timediff($t1, $t0)),"\n";
print "t2: ",timestr(timediff($t2, $t1)),"\n";
print "t3: ",timestr(timediff($t3, $t2)),"\n";
print "t4: ",timestr(timediff($t4, $t3)),"\n";
print "t5: ",timestr(timediff($t5, $t4)),"\n";


# Opens a dirhandle to read files, another to read sub-dirs and
# recursive calls itself foreach subdir it finds 
sub t1 {
    my $Dir = shift;
    opendir(DIR, $Dir) || die "Can't opendir $Dir: $!";
    my @Files = grep { /.txt/ && -f "$Dir/$_" } readdir(DIR);
    closedir DIR;
    opendir(DIR, $Dir) || die "Can't opendir $Dir: $!";
    my @Dirs = grep { /^[^.].*/ && -d "$Dir/$_" } readdir(DIR);
    closedir DIR;
    foreach $file (@Files) { print $Dir."-".$file."\n"; }
    foreach $SubDir (@Dirs) { &t1(join("\\",$Dir,$SubDir)); }
};

# Opens a dirhandle to read files, rewinds to read sub-dirs and
# recursive calls itself foreach subdir it finds 
sub t2 {
    my $Dir = shift;
    opendir(DIR, $Dir) || die "Can't opendir $Dir: $!";
    my @Files = grep { /.txt/ && -f "$Dir/$_" } readdir(DIR);
    rewinddir(DIR);
    my @Dirs = grep { /^[^.].*/ && -d "$Dir/$_" } readdir(DIR);
    closedir DIR;
    foreach $file (@Files) { print $Dir."-".$file."\n"; }
    foreach $SubDir (@Dirs) { &t2(join("\\",$Dir,$SubDir)); }
};

# Opens a dirhandle to read all directory contents and
# recursive calls itself foreach subdir it finds
# Uses File::Spec, which makes it portable
sub t3 {
    my ($Dir) = shift;
    my ($entry,@direntries,$fullpath);
    opendir( DIR, $Dir ) or die "Can't opendir $Dir: $!";
    @direntries = readdir( DIR ) or die "Error reading $Dir : $!\n";
    closedir DIR;
    foreach $entry (@direntries) {
        next if $entry =~ /^\.\.?$/;
        $fullpath = File::Spec->catfile( $Dir, $entry );
        if (-d $fullpath ) {
            &t3($fullpath);
        } elsif ( -f $fullpath && $entry =~ /.txt/) {
            print $Dir."-".$entry."\n";
        } 
    }
};

# Opens a dirhandle to read all directory contents and
# recursive calls itself foreach subdir it finds
sub t4 {
    my ($Dir) = shift;
    my ($entry,@direntries,$fullpath);
    opendir( DIR, $Dir ) or die "Can't opendir $Dir: $!";
    @direntries = readdir( DIR ) or die "Error reading $Dir : $!\n";
    closedir DIR;
    foreach $entry (@direntries) {
        next if $entry =~ /^\.\.?$/;
        $fullpath = join("\\",$Dir,$entry);
        if (-d $fullpath ) {
            &t4($fullpath);
        } elsif ( -f $fullpath && $entry =~ /.txt/) {
            print $Dir."-".$entry."\n";
        } 
    }
};

# Uses File::Find (whatever it does...)
sub t5 {
    my ($Dir) = shift;
    find(\&found, $Dir);
}

sub found {
    /.txt/ && print $File::Find::dir."-".$_."\n";
}

This test was run on a Pentiun 233 with 128Mb RAM, Windows 2000, FAT32 filesystem
C:\\Program holds 13477 files in 1206 folders of which 137 matches *.txt

t1: 27 wallclock secs ( 8.40 usr + 16.76 sys = 25.17 CPU)
t2: 24 wallclock secs ( 7.69 usr + 15.57 sys = 23.26 CPU)
t3: 47 wallclock secs (20.30 usr + 23.85 sys = 44.15 CPU)
t4: 36 wallclock secs (11.04 usr + 23.33 sys = 34.37 CPU)
t5: 30 wallclock secs (11.12 usr + 18.02 sys = 29.13 CPU)


/brother t0mas

[t0mas]

No offence taken. I think its a good thing to discuss/show different ways to solve the same problem, and I guess we all have our own toolkits of code snippets that we throw into every program we write.
Maybe I'll try to benchmark some of the ways when I'll find some time.

/brother t0mas