Depth Listing in Directory Traversal

[crabbdean]

For this I'll refer to a previous node 349582 and especially tye's response to that node 350948 in which he describes a "next" method iterator.

I've since used the guts of this "next" method (Big Thanks to tye) in a module I'm writing. I'm wanting to amend that method so that you can specify it to only go so deep into the directory tree. Here is the code:

sub this_deep {
    my ($self, $depth) = @_;
    $depth = -1 unless @_ > 1;
    $self->{depth} = $depth;
}

sub next {
    my $self = shift;
    while( 1 ) {
        if (  @{ $self->{output} }  ) {
            my $line = shift @{ $self->{output} };
            return $line;
        }
        if ( ! @{ $self->{dirs} } ) {
            return;
        }
        my $dir = shift @{ $self->{dirs} };
        if (-d $dir) {
            if(  opendir( DIR, $dir )  ) {
                map {
                    my $file = $_;
                    my $fullfile = File::Spec->catfile( $dir, $file );
                    if (-d $fullfile && ($self->{level} <= $self->{dep
+th} || $self->{depth} == -1)) {
                        push (@{ $self->{dirs} }, $fullfile);
                    }
                } File::Spec->no_upwards( readdir(DIR) );
                closedir DIR;
                
            } else {
                warn "\a\a\aopendir FAILED, $dir: $!";
            }
        } else {
            warn "\a\a\aNOT A DIRECTORY: $dir\n";
        }
    }
}

As you can see in this code I've put in the checks for the $self->{level} in that it will only push to the $self->{dirs} stack if the current level is below the specified depth or depth == -1 (if depth is set to -1 it will fully recurse the directory tree)

I'm having trouble building into this how to track the depth to ensure it only goes as far as specified. Any suggestions?

Update: I should add that what makes this tricky is the code makes it go DOWN a directory tree first before starting on the next level, not across the directory tree as is usually the case.

[tye]

Where you  push @{ $self->{dirs} },... you need to add push @{ $self->{levels} }, 1+$self->{level};. When you shift off of $self->{dirs}, also do $self->{level} = shift @{ $self->{levels} };.

You can do it in more memory-efficient ways but the added complexity seems not worth it.

[crabbdean]

tye you are great. :-) Love your work! ++ That worked a treat.

You know I was part way there in that thinking process, had similar thoughts but hadn't connected the pieces. Appreciate the help.

Thanks also to hbo that idea would have worked also but I felt this was cleaner.

[hbo]

sub next {
    my $self = shift;
    while( 1 ) {
        if (  @{ $self->{output} }  ) {
            my $line = shift @{ $self->{output} };
            return $line;
        }

Since we see no other references to this array ref, I'm assuming this code is cut down from its full version.

        if ( ! @{ $self->{dirs} } ) {
            return;
        }
        my $dir = shift @{ $self->{dirs} };
        if (-d $dir) {

Add a proper declaration for $olddir above, and:

          if ($oldir){
            $self->{level} += 
                ($dir=~tr!/!/!) - ($olddir=~tr!/!/!);
          }
          $olddir=$dir;

I haven't tested this, but the idea is that, since the paths are all absolute, a change in level means a difference in the number of path seperators. (You might want to use a more portable form of the seperator than a literal "/".)

[crabbdean]

Well I'm not sure this will work because the code makes the program go down the directory first, not across. Based on your idea it would have to keep a record somehow of the $oldir variable for when it finally finished going down the tree and came back to start on the next level. This requires it to keep a kind of memory of the tree and what it has or hasn't done. There must a simpler way.

BUT ... your idea has merit and something I'd considered also - your post has made me rethink it. If the number of path seperators is greater than $depth then don't push to the stack. That may work. I just don't like using relying on counting "\" in strings as a reliable method of keeping track of depth. If by chance a directory has that character in it, the routine will fail.

[hbo]

On Windows,"\" is the path seperator. A file name can't contain that character, so you won't encounter it unless it means a depth transition in the stack. Same goes for "/" on Unix. The tricky part with Windows would be a rooted path like "c:\\". For that you could count occurrences of "\+",although I believe that ActivePerl translates the seperators to "/" for portability with Unix.

The "width first" traversal of the directory tree doesn't matter in the code I gave. The path seperator count will be the same between peer directories, thus their difference will be zero, and we will add zero to the depth counter. This also works for transitions up the stack, since that difference will be negative.

[adamk]

In general, when doing queue-based-recursion, just unshift onto the queue, rather than pushing.



my @queue = ( $start ) {
while ( my $thing = shift @queue ) {
    do_something_with( $thing );
    unshift @queue, $thing->children;
}


I'm assuming this is what you are after?