Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical

Re^3: Finding files in one directory

by graff (Chancellor)
on Dec 18, 2012 at 08:40 UTC ( #1009313=note: print w/replies, xml ) Need Help??

in reply to Re^2: Finding files in one directory
in thread Finding files in one directory

If you're on a unix/linux system, running a "find" command in a subshell is okay, while perl's readdir will almost always do pretty much just as well in terms of performance, and I've seen one or two cases where perl does better. The nice thing about readdir is that you don't need to worry about possible artifacts in file names that affect the text output from "find" (e.g. it's possible to have things like line-feeds and carriage-returns embedded in file names).

Whenever I've tried to benchmark File::Find against unix "find" and simple (recursive) readdir, File::Find took noticeably longer to finish on relatively large directory structures. If you aren't dealing with nested directories, you don't need recursion, and readdir is definitely the easiest/best way to go.

BTW, the time needed to scan all the file names in a directory (or traverse a directory tree) is not affected by the quantity of data stored in the files; it's purely a matter of how many files per directory, and how many directories.

(The one case where a unix "find" command did worse that perl's "readdir" was on a ridiculously large directory - like a million files, all with fairly long names. Apparently, "find" (on a BSD system) was trying to hold the file names in memory, and at a certain point, it had to start using swap space, causing a geometric (exponential?) slow-down. Meanwhile, the run time for a simple perl script with  while($f=readdir(DIR)){...} was linear with the number of files, regardless of directory size.)

Replies are listed 'Best First'.
Re^4: Finding files in one directory
by nvivek (Vicar) on Dec 18, 2012 at 09:19 UTC
    Thanks for your deeper explanation. You given me what I expected. Thank you so much graff.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1009313]
[atcroft]: stevieb: I have berrybrew installed on a Win machine, and I noticed that with perlbrew you can "upgrade" a particular installed version... any chance of doing something similar in the
[atcroft]: future with berrybrew?
[atcroft]: stevieb: Also, one other odd question... is it possible to set it up so that all the versions installed can use a single directory that they pull module sources into, rather than duplicating files? (Am I making sense, or do I have too high a CBC
[atcroft]: (caffeine-blood content) atm?)
[atcroft]: .oO(Interesting... I just went to http://blogs.perl. org/, did a search for "Mars", and tried to go to the second page of results, and received instead a "Request-URI Too Large" message.... odd.)

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (6)
As of 2017-04-29 05:14 GMT
Find Nodes?
    Voting Booth?
    I'm a fool:

    Results (531 votes). Check out past polls.