Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris

Re^2: Finding files in one directory

by nvivek (Vicar)
on Dec 18, 2012 at 07:31 UTC ( #1009299=note: print w/replies, xml ) Need Help??

in reply to Re: Finding files in one directory
in thread Finding files in one directory

Thanks for your prompt reply Rolf. Will this readdir module will be better than find command ( performance wise )?

Replies are listed 'Best First'.
Re^3: Finding files in one directory
by graff (Chancellor) on Dec 18, 2012 at 08:40 UTC
    If you're on a unix/linux system, running a "find" command in a subshell is okay, while perl's readdir will almost always do pretty much just as well in terms of performance, and I've seen one or two cases where perl does better. The nice thing about readdir is that you don't need to worry about possible artifacts in file names that affect the text output from "find" (e.g. it's possible to have things like line-feeds and carriage-returns embedded in file names).

    Whenever I've tried to benchmark File::Find against unix "find" and simple (recursive) readdir, File::Find took noticeably longer to finish on relatively large directory structures. If you aren't dealing with nested directories, you don't need recursion, and readdir is definitely the easiest/best way to go.

    BTW, the time needed to scan all the file names in a directory (or traverse a directory tree) is not affected by the quantity of data stored in the files; it's purely a matter of how many files per directory, and how many directories.

    (The one case where a unix "find" command did worse that perl's "readdir" was on a ridiculously large directory - like a million files, all with fairly long names. Apparently, "find" (on a BSD system) was trying to hold the file names in memory, and at a certain point, it had to start using swap space, causing a geometric (exponential?) slow-down. Meanwhile, the run time for a simple perl script with  while($f=readdir(DIR)){...} was linear with the number of files, regardless of directory size.)

      Thanks for your deeper explanation. You given me what I expected. Thank you so much graff.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1009299]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (4)
As of 2017-02-28 06:51 GMT
Find Nodes?
    Voting Booth?
    Before electricity was invented, what was the Electric Eel called?

    Results (397 votes). Check out past polls.