Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?

File::Find problem

by Anonymous Monk
on Mar 17, 2003 at 16:30 UTC ( #243682=perlquestion: print w/replies, xml ) Need Help??

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello, monks. I'm trying to use the File::Find module to add a line to a file in a directory if that directory also contains a file with the extension .old. When I run the script, the preprocess and postprocess functions never execute. Here is the script:
#!/usr/bin/perl use File::Find; my $old_flag = 0; find ( { wanted => \&process, preprocess => \&preprocess, postprocess => \&postprocess }, '/home/greg/mydir' ); sub process { print "Processing\n"; if (/.*\.old$/) { $old_flag = 1; } } sub preprocess { print "Pre-Processing\n"; my @file_list = @_; $old_flag = 0; return @file_list; } sub postprocess { print "Post-Processing\n"; my $old_CPB = "CPB.old"; my $new_CPB = "CPB"; if ($old_flag == 1) { if (rename $new_CPB, $old_CPB) { open IN, $old_CPB or die "Unable to open input CPB: $!"; open OUT, ">$new_CPB" or die "Unable to open output CPB: $!"; while (<IN>) { chomp; if (/^cat.*/) { print OUT "cat\t/kat/src/all/bld_pipe.4go \\\n"; s/^cat\t/\t/; } print OUT "$_\n"; } } else { #not found return; } } close IN; close OUT; }
Can you help me figure out why the code isn't working as I expect? I am using Perl 5.005_03 on AIX.

Replies are listed 'Best First'.
Re: File::Find problem
by converter (Priest) on Mar 17, 2003 at 20:01 UTC

    It might be easier to process the directory tree in two passes. I'm thinking something like the following might work:

    #!/usr/bin/perl use warnings; use strict; use File::Find; my %targetdir; my %visiteddir; sub wanted_stage1 { return unless -f; $targetdir{$File::Find::dir}++ if /\.old$/; } sub postprocess { return unless $targetdir{$File::Find::dir}; return if $visiteddir{$File::Find::dir}++; print "Changing files in $File::Find::dir\n"; } # build list of directories containing *.old files: find({wanted => \&wanted_stage1}, "."); print "Directories to process: @{[sort keys %targetdir]}\n"; # process files in directory list: find({wanted => sub{}, postprocess => \&postprocess}, sort keys %targe +tdir); __END__ .: code.txt one two ./one: bar foo three ./one/three: four something.old ./one/three/four: blah something.old ./two: bar foo something.old Directories to process: ./one/three ./one/three/four ./two Changing files in ./one/three/four Changing files in ./one/three Changing files in ./two
Re: File::Find problem (yucky callbacks)
by tye (Sage) on Mar 19, 2003 at 08:19 UTC

    As I've just been ranting in another thread: callbacks aren't a great interface (to say the least).

    I find File::Find so quirky to use that it usually takes me less time to write my own directory scanner than it does to figure out how to get File::Find to do what I want. Sure, for simple "do this to nearly every file" operations, File::Find can save me time. But for those I tend to use /bin/find instead anyway (from cygwin or find2perl if needed).

    There are a few classic mistakes you need to keep in mind when searching a directory tree:

    • Don't follow symbolic links unless you either can guarentee that there are no loops or you do the work to detect them.
    • If you use opendir on anything but "." [ or, File::Spec::curdir() for those being portable -- which doesn't include File::Find, but "." is pretty portable (: ], then you can't stat the results unless you prepend the directory path to the front first [ or use File::Spec::catfile() -- again unlike File::Find ]
    • Using chdir allows you to opendir "." and is more efficient so remember to chdir("..") [ er, chdir( File::Spec::updir() ) ]
    • Don't recurse into "." nor ".." [curdir()/updir()]
    • glob is often much easier than opendir but it ignores ".*" files unless you tell it not to.
    So here is my way of doing it which I think is much more natural:
    #!/usr/bin/perl -w use strict; Scan( '/home/greg/mydir' ); sub Scan { my( $dir, $path )= @_; $path ||= "."; chdir $dir or die "Can't chdir($dir) from $path: $!\n"; $path= $dir if "." eq $path; my @files= ( glob("*"), grep "." ne $_ && ".." ne $_, glob(".*") ); for my $sub ( grep ! -l $_ && -d _, @files ) { Scan( $sub, "$path/$sub" ); } if( grep /\.old$/, @files ) { local( @ARGV )= "CPB"; local( $^I )= ".old"; while( <> ) { if( /^cat/ ) { print "cat\t/kat/src/all/b-ld_pipe.4go \\\n"; s/^cat\t/\t/; } print; } } chdir ".."; }

                    - tye
      There is File::Iterator, I have not used it yet, but it seems to simplify things.

        Actually, I've been thinking about this and I think my preferred interface might be an object that lets you iterate in several directions so you can pick depth-first or bredth-first and decide to prune your search in what I consider more natural ways:

        # The simple case like File::Find / File::Iterator my $iter= File::Whatever->new( $root ); while( $iter->NextItem() ) { doSomething( $iter->GetFileName() ); } # Recurse yourself: my $iter= File::Whatever->new( $root ); MyFunc( $iter ); sub MyFunc { my $iter= shift(@_); # Uncomment next line for depth-first search: MyFunc( $iter ) while $iter->PushPath(); while( my $file= $iter->NextFile() ) { doSomething( $file ); } # Uncomment next line for bredth-first search: #MyFunc( $iter ) while $iter->PushPath(); $iter->PopPath(); } # Example of pruning your search: my $iter= File::Whatever->new( $root ); MyFunc( $iter ); sub MyFunc { my $iter= shift(@_); while( my $file= $iter->NextFile() ) { doSomething( $file ); } while( my $sub= $iter->PushPath() ) { if( $sub =~ /debug/i ) { $iter->PopPath(); } else { MyFunc( $iter ); } } $iter->PopPath(); } # Bredth-first search w/o recursion: my $iter= File::Whatever->new( $root ); do { while( my $file= $iter->NextFile() ) { doSomething( $file ); } } while( $iter->NextPath() ); # Bredth-first search w/ pruning w/o recursion: my $iter= File::Whatever->new( $root ); do { while( my $file= $iter->NextFile() ) { doSomething( $file ); } while( $iter->NextPath() =~ /debug/i ) { $iter->PopPath(); } } while( ! $iter->Finished() ); # Depth-first search w/o recursion: my $iter= File::Whatever->new( $root ); do { 0 while $iter->PushPath(); while( my $file= $iter->NextFile() ) { doSomething( $file ); } } while( $iter->PopPath() ); # Depth-first search w/ pruning w/o recursion: my $iter= File::Whatever->new( $root ); do { while( my $sub= $iter->PushPath() ) { $iter->PopPath() if $sub =~ /debug/i; } while( my $file= $iter->NextFile() ) { doSomething( $file ); } } while( $iter->PopPath() ); # The opposite of pruning: # Only do things to files in directories named "debug" my $iter= File::Whatever->new( $root ); do { if( 'debug' eq $iter->GetDirName() ) { while( my $file= $iter->NextFile() ) { doSomething( $file ); } } } while( $iter->NextPath() );
        But I need to think about that a lot more before I'm sure that such makes sense or if I can make it better. (:

                        - tye
Re: File::Find problem
by pg (Canon) on Mar 17, 2003 at 17:06 UTC
    I tried your code with a small modification, and it worked. The second parameter to find() should be an array, so I modified it to this:
    find ( { wanted => \&process, preprocess => \&preprocess, postprocess => \&postprocess }, (".") );

    I understood your question, and the print out did show me that all three handler were executed. Hm...


      I tried your change, and it still doesn't work for me. Thanks for the attempt though.

      Also, In case I didn't make it clear in my initial question, the process function IS executing. The preprocess and postprocess functions are not.

Re: File::Find problem
by arturo (Vicar) on Mar 17, 2003 at 17:41 UTC

    I have a suggestion about a redesign that might help (although I'll admit I haven't figured out why your code isn't working -- one guess is that you're reading the perldocs for the wrong version of File::Find, but I don't know off the top of my head at which version the pre- and post- process options were added, but defeasible memories suggest it is VERY recent; check perldoc File::Find on your own system to see if your version supports these options).

    My suggestion is this: do away with the flag variable, and just have your "wanted" routine process the directory itself.

    sub process { return unless -d; opendir DIR, $_ or die "Can't open $File::Find::name: $!\n"; my @old = grep /\.old$/i, readdir DIR; closedir DIR; return unless @old; # now write what you have to }

    This is a little bit inefficient (it ends up reading the directory twice), but I think it gains a bit in clarity. If my guess above is right, it will work whereas your current algorithm won't unless you upgrade perl (or at least File::Find) on your system.

    There are a few things you're doing that don't comport with what you say you want to do. Do you want to append lines to the CPB files or do you want to write new ones? Also, that else in the postprocess sub is unnecessary.

    update shoulda checked the File::Find docs .. .the pre and postprocess facility has been there for a very long time, so my guess is wrong. Musta blocked that part of the docs out. Also, go ahead and disregard the comment about "appending", I hadn't read that part of your code very closely.

    If not P, what? Q maybe?
    "Sidney Morgenbesser"

Re: File::Find problem
by clairudjinn (Beadle) on Mar 17, 2003 at 17:44 UTC

        preprocess(): you pass in a list of files, set a flag, and then pass the same list back...seems to me that all you need to do is set the flag...

        preprocess() is intended to be used as a filter for the list of files returned by readdir(), so it is expected to return a list, which in turn replaces the original list returned by readdir().

        Eliminating the unnecessary assignment, preprocess() could be reduced to:

        sub preprocess { print "Pre-Processing\n"; $old_flag = 0; @_; }

        This would probably be a little more efficient, but I'd be inclined to look for an even more efficient solution that would eliminate the need to pass the entire file list around just to set a flag.

    Re: File::Find problem
    by Aristotle (Chancellor) on Mar 20, 2003 at 20:26 UTC

      You are probably using too old a version of File::Find. Pre- and postprocess callbacks are a somewhat recent addition that entered the fray around Perl 5.6 IIRC.

      A somewhat cleaned up version of your script:

      #!/usr/bin/perl -w use strict; use constant OLD => "CPB.old"; use constant NEW => "CPB"; use File::Find; my $old_flag = 0; find ( { preprocess => sub { print "Pre-Processing\n"; $has_old = grep /\.old$/, @_; @_; }, wanted => sub {}, postprocess => sub { print "Post-Processing\n"; return unless $has_old; return unless rename NEW, OLD; open my $in, "<", OLD or die "Unable to open input CPB: $!"; open my $out, ">", NEW or die "Unable to open output CPB: $!"; while (<$in>) { s!^cat\t!cat\t/kat/src/all/bld_pipe.4go \\\n\t!; print $out $_; } } }, '/home/greg/mydir' );

      Makeshifts last the longest.

    Log In?

    What's my password?
    Create A New User
    Domain Nodelet?
    Node Status?
    node history
    Node Type: perlquestion [id://243682]
    Approved by broquaint
    Front-paged by broquaint
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others chilling in the Monastery: (2)
    As of 2023-10-02 04:50 GMT
    Find Nodes?
      Voting Booth?

      No recent polls found