Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Re: File::Find memory leak

by graff (Chancellor)
on Jan 27, 2004 at 14:17 UTC ( #324424=note: print w/ replies, xml ) Need Help??


in reply to File::Find memory leak

I don't mean to spoil the fun of using perl, but in a case like this, I would consider looking at a Windows port of the GNU find utility. It will undoubtedly be faster and have a smaller memory footprint. (Frankly, the File::Find module seems to be a fountain of difficulty... I tend to avoid it.)


Comment on Re: File::Find memory leak
Re: Re: File::Find memory leak
by crabbdean (Pilgrim) on Jan 30, 2004 at 05:25 UTC
    Thanks graff, this is a clever solution. I'm all up for looking at alternatives. Am looking into this now. Likewise I'm also tending to avoid the File::Find module. I'm having to rewrite my program now as the use of the File::Find module was at the heart of it, and its rendered my program obsolete as a practical solution due to the sheer size of our file server.

    Thanks mate
    Dean
      Just a thought about something you might try... This works for me under unix, and I expect it would work in windows as well. It's very good in terms of using minimal memory, and having fairly low system overhead overall:
      chdir $toppath or die "can't cd to $toppath: $!"; open( FIND, "find . -type d |" ) or die "can't run find: $!"; while ( my $d = <FIND> ) { chomp $d; unless ( opendir( D, $d )) { warn "$toppath/$d: open failed: $!\n"; next; } while ( my $f = readdir( D )) { next if ( -d "$d/$f" ); # outer while loop will handle all dir +s # do what needs to be done with data files } # anything else we need to do while in this directory } close FIND;
      This has the nice property that all the tricky recursion stuff is handled by "find", while all the logic-instensive, file-based stuff is handled pretty easily by perl, working with just the data files in a single directory at any one time.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://324424]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (6)
As of 2014-10-21 02:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    For retirement, I am banking on:










    Results (95 votes), past polls