Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Re: Splitting up a filesystem into 'bite sized' chunks

by sundialsvc4 (Abbot)
on Jul 12, 2013 at 04:37 UTC ( #1043872=note: print w/ replies, xml ) Need Help??


in reply to Splitting up a filesystem into 'bite sized' chunks

Maybe I should adopt the principle of writing every single terse-comment that I am prone to in a splendiferous loquacious paragraph, or three, in a vain attempt to forestall the “down-vote demons.”   I dunno.   But, wrapped-up in the terse-comment “NFS is a monster” is a very-valid point:   NFS is a network file-system that does not (unlike, say, Microsoft’s famous system) pretend to be otherwise.

With NFS, filesystems can be unfathomably-large, and network transports can be slow, and NFS will still work.   However, all that having been said ... your (Perl-implemented) algorithms must match.   You must, for example, come up with a plausible strategy for “splitting up a filesystem into bite-sized chunks,” whatever that strategy might be, that assumes both that you cannot immediately ascertain how many files/directories are in any particular area of that filesystem, and that you cannot obtain such a count in a timely fashion.   Instead of an algorithm, therefore, you are obliged to make use of a heuristic.


Comment on Re: Splitting up a filesystem into 'bite sized' chunks
Re^2: Splitting up a filesystem into 'bite sized' chunks
by Preceptor (Chaplain) on Jul 12, 2013 at 22:23 UTC

    NFS does have it's limitations - one of these is transport layer - you can do 10G multi channel Ethernet but you still have a price to pay with the connection latency. With a large storage environment you get a lot of spindles and controllers, but that only helps if you can go wide on your IO

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1043872]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (13)
As of 2015-07-06 14:12 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (75 votes), past polls