Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"

fast count files

by gautamparimoo (Beadle)
on Jan 06, 2012 at 07:35 UTC ( #946545=perlquestion: print w/replies, xml ) Need Help??
gautamparimoo has asked for the wisdom of the Perl Monks concerning the following question:

I have tried File::Name and readdir to count file in the drive but results are slow when disk size increases. Iz there a way to just give total count of files in drive very quickly(eg 10 mins for 200gb)? Help is appreciated

Replies are listed 'Best First'.
Re: fast count files
by BrowserUk (Pope) on Jan 06, 2012 at 08:17 UTC

    What OS? Which filesystem?

      Windows os NTFS filesystem

        If you literally just want to count the files on the entire disk, this is by far the fastest simple method I know of.

        It counts the 1.2 million files on my cold-cache, 640GB (400GB used) drive in a little under 7 minutes:

        $t=time; $n = `attrib /s c:\\* | wc -l`; printf "$n : %.f\n", time()-$t;; 1233597 : 394

        Try it and see how you fare. I vaguely remember finding a faster method years ago, and I'll try to remember enough to look it up.

        Note: Don't do my @files = `attrib /s c:\\*`; my $n = scalar @files; All the memory allocation slows things down horribly.

        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

        The start of some sanity?

Re: fast count files
by umasuresh (Hermit) on Jan 09, 2012 at 12:53 UTC
    Try GNU parallel
    find /current_dir/ -type f | parallel -k -j150% -n 1000 -m stat -c %s + > sizeslist && awk '{sum+=$1}END{print "Total Bytes:",sum}' sizeslis +t
Re: fast count files
by spx2 (Deacon) on Jan 09, 2012 at 11:19 UTC
    If your only concern is speed I think you should rewrite it in C. (First place I'd start looking is at the source of du or df)

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://946545]
Approved by planetscape
Front-paged by toolic
[1nickt]: But the bands are even louder! I saw Spearhead (Michael Franti) at an outdoor show and had to walk a mile away to not feel pain in my chest! Babies were crying ... I asked the sound engineer why it was necessary to have the bass so loud and he laughed...
[Discipulus]: but the best i attended live was Mano Negra Patchanka at Forte Prenestino .. in 1990
[Corion]: Hmmm - Mano Negra or at least Manu Chao seem to put on a good live show. At least the one live CD I have from Manu Chao sounds good ;)
Discipulus feels the same jealousity of the johngg's daughter
[1nickt]: choroba I agree
[choroba]: Playing in a punk rock band for 20 years... my hearing is quite bad
[Corion]: I still have hopes to turn my godson and his two siblings into a punk band ;)
[Corion]: Their older sister just started piano but has been interested in drumming, which she should be able to start with 8 years or so)
[Discipulus]: ah was Kink of Bongo 1992..

How do I use this? | Other CB clients
Other Users?
Others meditating upon the Monastery: (9)
As of 2017-03-24 12:21 GMT
Find Nodes?
    Voting Booth?
    Should Pluto Get Its Planethood Back?

    Results (301 votes). Check out past polls.