Beefy Boxes and Bandwidth Generously Provided by pair Networks Frank
Syntactic Confectionery Delight
 
PerlMonks  

Re: fast count files

by BrowserUk (Pope)
on Jan 06, 2012 at 08:17 UTC ( #946550=note: print w/ replies, xml ) Need Help??


in reply to fast count files

What OS? Which filesystem?


Comment on Re: fast count files
Re^2: fast count files
by gautamparimoo (Beadle) on Jan 06, 2012 at 08:50 UTC
    Windows os NTFS filesystem

      If you literally just want to count the files on the entire disk, this is by far the fastest simple method I know of.

      It counts the 1.2 million files on my cold-cache, 640GB (400GB used) drive in a little under 7 minutes:

      $t=time; $n = `attrib /s c:\\* | wc -l`; printf "$n : %.f\n", time()-$t;; 1233597 : 394

      Try it and see how you fare. I vaguely remember finding a faster method years ago, and I'll try to remember enough to look it up.

      Note: Don't do my @files = `attrib /s c:\\*`; my $n = scalar @files; All the memory allocation slows things down horribly.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

      The start of some sanity?

        You might have issues using wc -l on Windows, unless you have installed it from somewhere like GNU. However it is easy to implement your own in Perl.
        Based on BrowserUk's ideas:
        use strict; use warnings; my $t=time; # $n = `attrib /s c:\\* | wc -l`; open (my $pipe, '-|', 'attrib /s c:\\*') or die "attrib: $!"; my $n = 0; while (<$pipe>) { $n++ } close $pipe; printf "$n : %.f\n", time()-$t;
        Gives: 380366 : 135 on a used space of 297GB.

        Thanks for that quick reply But I have 2 use my @files = `attrib /s c:\\*`; my $n = scalar @files; as without it says that wc is an unrecognised command .Also the I am showing the code of the fastest method I know

        use strict; my $f; # number of files my $d; # number of dirs sub count_files { my ($ref) = @_; foreach my $dir (@$ref) { $dir = readlink $dir and chop $dir if -l $dir; # read link next unless opendir(my $dir_h, $dir); # open dir o ++r next my @dirs; while (defined(my $file = readdir $dir_h)) { if ($file eq '.' or $file eq '..') { next; } if (-d "$dir/$file") { ++$d; # counting d +irs push @dirs, "$dir/$file"; } elsif(-f _) { ++$f; # counting f +iles } } closedir $dir_h; count_files(\@dirs); } [$f, $d]; } foreach my $arg (@ARGV) { my @dir = -d $arg ? $arg : next; ($f, $d) = (0, 0); print "$arg\nFiles\t: $$_[0]\nFolders\t: $$_[1]\n" for count_files +(\@ dir); }

        suggest a faster method and also how to run without using my @files = `attrib /s c:\\*`; my $n = scalar @files;

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://946550]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (13)
As of 2014-04-18 17:03 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (471 votes), past polls