Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Re: Efficiency: Recursive functions while de-subbing my program

by zork42 (Monk)
on Jul 13, 2013 at 10:35 UTC ( #1044141=note: print w/ replies, xml ) Need Help??


in reply to Efficiency: Recursive functions while de-subbing my program

4 points:

===

(1) I'm not sure you've structured your code correctly.

Normally you don't use File::Find recursively.

The typical use is:

use File::Find; find(\&wanted, @directories_to_search); sub wanted { ... }
ie you make 1 single call to find().
Then find() calls the provided callback function wanted() for each file and directory it finds as it searches the provided directories.

In your case you might have:
use File::Find; find( { wanted => \&_process_files, no_chdir => 1 }, @directories_to_ +search); sub _process_files { my $file = $File::Find::name; # .json file if (-f $file and ($file =~ /\.json\Z/)) { _process_json($file); } }

===

(2) Is your regexp correct?

If you're interested in files with an extension of ".json", then it should look like this: $file =~ /\.json$/i

===

(3) As everyone has said, in this case the function call time is tiny compared with the disk I/O time.

===

(4) What I normally do if processing the files will take significantly more time than finding the files is this:
  1. find all the files of interest and store them in an array
  2. process each file in the array

The advantage of this is in step 2 you can provide the user with feedback like:
"Currently processing file 123 of 12345 files. Estimated time to finish is xxxx".

To provide an accurate ETA, if the time to process a file is proportional to the file's size, you would need to save the size of each individual file as it is found in step 1.


Comment on Re: Efficiency: Recursive functions while de-subbing my program
Select or Download Code
Re^2: Efficiency: Recursive functions while de-subbing my program
by choroba (Abbot) on Jul 13, 2013 at 11:00 UTC
    if the time to process a file is proportional to the file's size, you would need to save the size of each individual file
    Only if the files are not sized similarly.
    لսႽ ᥲᥒ⚪⟊Ⴙᘓᖇ Ꮅᘓᖇ⎱ Ⴙᥲ𝇋ƙᘓᖇ
      That true.

      But the general case is so easy to implement, you might as well be general :)

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1044141]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (16)
As of 2014-12-22 17:31 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (124 votes), past polls