Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Need efficient way to fetch 20 last modified files from folder

by anykeyman (Novice)
on Aug 24, 2012 at 19:57 UTC ( #989630=perlquestion: print w/ replies, xml ) Need Help??
anykeyman has asked for the wisdom of the Perl Monks concerning the following question:

Code like  map {(stat $_)[9]} readdir($handle) takes up to 2-3 seconds, because there are up to tens thousands files in some folders.

I guess there is some index by mdate in file system, so it is possible to fetch 20 last modified files efficiently.

How to do so?

(I need to run script on various Windows servers, no UNIX. So my guess is to use some WINAPI calls?)

Comment on Need efficient way to fetch 20 last modified files from folder
Download Code
Re: Need efficient way to fetch 20 last modified files from folder
by Anonymous Monk on Aug 24, 2012 at 22:55 UTC
    chomp(my @newest = (`dir /A-D /O-D /D`)[5 .. 24]); # lines 0 - 4 - hea +der say for @newest;
    If for Windows only, the switches,

    /A-D files only /O-D ordered from newest to oldest /D output in column

    Don't know if this will be faster for you. Type help dir at your command prompt.

Re: Need efficient way to fetch 20 last modified files from folder
by GrandFather (Cardinal) on Aug 25, 2012 at 00:30 UTC
    I guess there is some index by mdate in file system

    Then most likely you guess wrong. Even if such an index exists in some versions of Windows for some disk format, it is not universally true.

    Your best bet if you can is to restructure the directory layout so that you don't end up with large numbers of files per folder and can efficiently find stuff. Even better, if you have end to end control of this system, use a database for storing the file locations, touch times and other interesting meta data.

    Another alternative is to use the file change notification system Windows provides to maintain a database of file touch times independently of the OS.

    True laziness is hard work
Re: Need efficient way to fetch 20 last modified files from folder
by rpnoble419 (Pilgrim) on Aug 25, 2012 at 01:46 UTC

    I have used this in the past to monitor when a file was added to a directory for printing. Win32::FileSystem::Watcher

    I was able to modify the example code to perform the print task I needed.

Re: Need efficient way to fetch 20 last modified files from folder
by philiprbrenan (Monk) on Aug 30, 2012 at 18:51 UTC

    I think you would have to descend into C and use Windows API calls directly to get a further performance improvement. However, the cost to you in your time of doing such a project is unlikely to be repaid from the time saved. 2-3 seconds sounds pretty good on that basis.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://989630]
Approved by moritz
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (4)
As of 2014-08-02 03:30 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Who would be the most fun to work for?















    Results (54 votes), past polls