Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

Re^2: Random Files Recursively Form Directory

by gautamparimoo (Beadle)
on Apr 02, 2013 at 05:07 UTC ( #1026580=note: print w/replies, xml ) Need Help??


in reply to Re: Random Files Recursively Form Directory
in thread Random Files Recursively Form Directory

Yes the % is only for the target files(ie 25% should be the extensions specified). Next the data structure would become very big as I have drives with minimum 100GB that means 25Gb(25%) minimum so the time would be quite high (Time for file::find and Filelist prparation + shuffle of filelist). So is there another workaround?

I saw this module File::Random which implements rand functions. I was thinking of using this module in a while loop to iterate over the directory till specified size is encountered. What do you think of this?

One machine can do the work of fifty ordinary men. No machine can do the work of one extraordinary man. -Elbert Hubbard
  • Comment on Re^2: Random Files Recursively Form Directory

Replies are listed 'Best First'.
Re^3: Random Files Recursively Form Directory
by CountOrlok (Friar) on Apr 02, 2013 at 13:01 UTC

    I don't see how the size of the files in a directory would affect File::Find, data structure preparation and shuffling the list. I would say that number of files in a directory would have an effect. What is the number of files per directory?

    I would think File::Random would also be slow if the number of files in a directory is huge. A quick check of the code in File::Random make me think that it re-reads the whole directory every time you call random_file() and therefore you could get the same file returned more than one time.

      Each directory consists of atleast 5000 files plus sub directories having atleast 3000 files. But any how it looks like the only possible solution so ill give it a try. Thnks...

      One machine can do the work of fifty ordinary men. No machine can do the work of one extraordinary man. -Elbert Hubbard

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1026580]
help
Chatterbox?
[hippo]: Make it an option?
[Corion]: hippo: Yes, but I'm not sure that it's even worth the effort of implementing it at all...
[Corion]: You'll only ever need that option if you have a long-running query whose results are not cached by your DB already, and in those cases I presume that the programmer will want to maintain the temporary tables themselves - I wouldn't know when to drop ...
[Corion]: ... the temporary tables, and also don't have a good idea on how to create unique table names for them
[hippo]: OIC. In that case leave it out but invite feature requests and see if any of the users suggest it. :-)
[Corion]: Talking about this makes me realize that it's likely only a half useful idea. But it still would be convenient to have as an option...
[Corion]: hippo: Hmmm - yeah, I could document it and wait for code implementing that option to show up ;-D

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (10)
As of 2017-02-23 15:34 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Before electricity was invented, what was the Electric Eel called?






    Results (347 votes). Check out past polls.